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Abstract 


Acquiring  defense  systems  in  the  face  of  urgent  needs,  budget  challenges 
and  scarce  resources  drives  the  need  for  greater  process  efficiency,  placing  focus 
on  designing  for  affordability  as  a  requirement.  While  there  is  progress,  an  efficient 
process  for  early  lifecycle  affordability  tradeoff  that  takes  uncertainty  into  account  is 
lacking.  Methods  for  exploring  design  tradespaces  have  matured,  but  largely  fall 
short  in  evaluating/selecting  resilient  system  concepts,  meaning  affordably 
adaptable  under  uncertain  futures.  Affordability  tradeoffs  are  possible,  but  limited  to 
tradeoffs  for  current  operating  environments  or  single  point  futures.  As  such,  it’s  not 
possible  to  truly  evaluate  system  concepts  for  resiliency  since  this  necessitates 
evaluation  across  many  alternative  futures  that  may  unfold  differently.  A  recently 
developed  approach,  Epoch-Era  Analysis,  provides  the  ability  to  evaluate  system 
concepts  for  both  multiple  epochs  (periods  of  fixed  context  and  needs)  and  eras 
(ordered  sequences  of  epochs).  This  research  has  demonstrated  usefulness  of 
Epoch-Era  Analysis  in  time-related  affordability  tradeoffs,  with  concurrent 
consideration  of  affordability  with  other  key  decision  criteria,  such  as  performance 
uncertainty.  Short-run  and  long-run  impacts  of  decisions  in  the  face  of  temporally 
unfolding  uncertainties  are  explicitly  addressed  by  encapsulating  key  uncertainty 
factors  into  epochs  and  eras.  A  beta-level  affordability  tradeoff  method  with  case 
application  has  been  developed  in  this  one-year  research  study.  The  method 
enables  analysis  of  affordability  in  the  face  of  changing  contexts  and  needs  (epochs) 
in  short-run  and  long-run  alternative  futures  (eras).  In  support  of  early  lifecycle 
affordability  tradeoffs,  the  method  supports  decision-makers  faced  with  choosing 
among  multiple  alternative  system  concepts  to  answer  the  question,  Which  system 
concept  delivers  highest  capabilities  at  cost  over  time,  taking  into  account  potential 
shifts  in  context  (e.g.,  policy  change,  technology  availability)  and  needs  (e.g., 
change  in  mission)? 


Keywords:  affordability,  design  tradespaces,  trade-offs,  uncertainty,  epoch-era 
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Executive  Summary 


Acquiring  defense  systems  in  the  face  of  urgent  needs,  budget  challenges, 
and  scarce  resources  drives  the  need  for  greater  process  efficiency.  A  recent 
strategy  to  address  these  challenges  focuses  on  designing  for  affordability  as  a 
requirement.  While  there  is  progress,  an  efficient  process  for  early  lifecycle 
affordability  tradeoff  that  takes  uncertainty  into  account  has  been  lacking. 

Methods  for  exploring  design  tradespaces  have  matured,  but  largely  fall  short 
in  evaluating/selecting  resilient  system  concepts,  meaning  affordably  adaptable 
under  uncertain  futures.  Affordability  tradeoffs  are  possible  but  limited  to  tradeoffs  for 
current  operating  environments  or  single  point  futures.  As  such,  it’s  not  possible  to 
truly  evaluate  system  concepts  for  resiliency  because  this  necessitates  evaluation 
across  many  alternative  futures  that  may  unfold  differently.  A  recently  developed 
approach,  Epoch-Era  Analysis,  provides  the  ability  to  evaluate  system  concepts  for 
both  multiple  epochs  (periods  of  fixed  context  and  needs)  and  eras  (ordered 
sequences  of  epochs).  Epochs  and  eras  are  generated  though  parameterization  of 
uncertainty  variables  (e.g.,  available  technology,  missions).  The  approach  has  been 
applied  to  adaptability  tradeoffs  in  several  case  studies. 

This  research  has  demonstrated  usefulness  of  Epoch-Era  Analysis  in  time- 
related  affordability  tradeoffs,  with  concurrent  consideration  of  affordability  with  other 
key  decision  criteria,  such  as  performance  uncertainty.  Short-run  and  long-run 
impacts  of  decisions  in  the  face  of  temporally  unfolding  uncertainties  are  explicitly 
addressed  by  encapsulating  key  uncertainty  factors  into  epochs  and  eras.  A  beta- 
level  affordability  tradeoff  method  with  case  application  has  been  developed  in  this 
one-year  research  study.  The  method  enables  analysis  of  affordability  in  the  face  of 
changing  contexts  and  needs  (epochs)  in  short-run  and  long-run  alternative  futures 
(eras).  Expected  impact  is  a  means  for  improving  decisions  in  designing  for 
affordability  as  a  requirement  in  support  of  better  buying  power,  ultimately  yielding 
benefits  to  both  warfighters  and  taxpayers.  The  ability  to  perform  affordability 
tradeoffs  early  in  the  lifecycle,  where  forward-looking  uncertainty  is  highest,  is 
essential  to  meeting  this  goal. 

In  support  of  early  lifecycle  affordability  tradeoffs,  the  proposed  method 
empowers  decision-makers  faced  with  choosing  among  multiple  alternative  system 
concepts  to  answer  the  question,  Which  system  concept  delivers  highest  capabilities 
at  cost  over  time,  taking  into  account  potential  shifts  in  context  (e.g.,  policy  change, 
technology  availability)  and  needs  (e.g.,  change  in  mission)? 

This  research  project  has  made  progress  in  addressing  the  two  challenges 
identified  in  the  literature  review  and  conversations  with  selected  experts.  The  first 
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challenge  is  lack  of  an  accepted  definition  of  affordability.  The  research  resulted  in  a 
proposed  definition  for  affordability  and  affordable  design  solution,  and  proposed 
affordability  analysis  as  identification  of  solutions  that  remain  affordable  in  short  run 
and  long  run.  The  second  challenge  is  lack  of  metrics  with  systematic  framework. 
The  research  resulted  in  a  proposed  method,  incorporating  Multi-Attribute 
Tradespace  Exploration  (MATE),  Epoch-Era  Analysis  (EEA)  and  Multi-Attribute 
Expense  (MAE).  The  method  was  applied  to  a  demonstration  case  for  a  Next- 
Generation  Combat  Ship  (NGCS). 

The  research  resulted  in  a  published  paper  for  the  2013  Naval  Postgraduate 
School  Acquisition  Research  Symposium  and  the  submission  of  a  paper  abstract  for 
a  future  conference  (accepted). 
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Affordability  Tradeoffs  Under  Uncertainty 
Using  Epoch-Era  Analysis 


Literature  Review 

This  section  presents  a  review  of  selected  literature  for  key  affordability 
concepts  and  motivations  for  design  for  affordability.  As  various  research  methods 
have  been  used  to  frame  affordability  as  a  new  construct  within  systems 
engineering,  their  effectiveness  or  deficiencies  in  managing  cost  and  schedule 
reductions  is  discussed.  Finally,  emerging  methods  are  introduced  as  possible 
approaches  for  enabling  the  design  for  affordability  through  explicit  considerations  of 
time  dependencies  and  uncertainty  during  early-phase  decision-making. 

Rising  Cost  Growth  and  Shrinking  Defense  Budgets 

In  recent  decades,  the  cost  growth  of  defense  weapon  systems  has  been 
escalating  (Department  of  Defense  [DoD],  2013),  posing  ever-greater  obstacles  to 
the  defense  acquisition  process.  Despite  numerous  successes,  historical  records 
have  shown  that  many  programs  have  been  plagued  by  massive  cost  overruns, 
schedule  delays,  failure  to  anticipate  future  requirements,  and  ultimately  unrealized 
capabilities  (Cordesman  &  Frederiksen,  2006).  These  failures  threaten  the  economic 
feasibility  of  future  acquisition  programs  and  combat  readiness  of  the  U.S.  military  in 
the  long  run. 

Many  investigations  have  been  conducted  into  major  causes  of  cost  and 
schedule  growth  in  defense  acquisition.  A  2009  report  by  Porter  et  al.  has  narrowed 
the  causes  to  two  main  categories:  weaknesses  in  management  visibility,  direction, 
and  oversight;  and  weaknesses  in  initial  program  definition  and  costing  (Porter  et  al., 
2009).  The  second  category  encompasses  failures  in  systems  design  and  early- 
phase  planning,  as  well  as  unrealistic  cost  estimates.  Specific  causes  within  this 
category  may  include  failure  in  eliciting  or  anticipating  stakeholder  requirements,  use 
of  immature  technologies,  inadequate  systems  engineering,  and  inefficiencies 
resulting  from  schedule  compression.  Shrinking  defense  budgets  over  the  years  may 
also  render  many  acquisition  programs  unaffordable  in  future.  Recent  annual 
budgets  have  shifted  in  scope  and  focus  as  they  attempt  to  reduce  acquisition  costs, 
make  better  usage  of  resources,  and  achieve  better  buying  power  (Office  of  the 
Under  Secretary  of  Defense  [Comptroller],  2013).  Defense  budgets  of  2010  and 
2011  were  primarily  allocated  to  the  termination  of  weapons  programs  experiencing 
high  cost  and  schedule  overruns,  while  the  budgets  of  2012  and  2013  have  shifted 
to  refining  defense  business  operations.  These  budgets  aim  to  achieve  more  lean 
acquisition  programs  with  reduced  overhead  and  support  costs.  Most  significant 
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among  these  refinements  is  the  implementation  of  the  Better  Buying  Power  (BBP) 
initiative  (Carter,  2010a),  aiming  to  restore  affordability  through  pursuing  greater 
efficiencies  and  responsiveness  in  acquisition. 

The  BBP  initiative  offers  guidance  to  the  acquisition  community  for  obtaining 
greater  efficiency  and  productivity  in  defense  spending.  Apart  from  recommending 
strategy-driven  changes  in  labor  force  structure  and  modernization,  it  emphasizes 
the  more  disciplined  use  of  resources.  Central  to  the  streamlining  of  business 
operations  is  the  principle  of  targeting  affordability  and  controlling  cost  growth  in 
acquisition  programs.  With  rising  cost  growth  and  shrinking  defense  budgets, 
Congress  and  policymakers  are  compelled  to  mandate  affordability  as  a  requirement 
at  all  milestone  decision  points  of  program  development  (Carter,  2010a,  2010b). 

Introduction  of  Affordability  in  Defense  Acquisition 

Following  the  BBP  initiative  and  the  budgetary  realities  facing  the  defense 
industry,  Dr.  Ashton  Carter,  then  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (USD[AT&L]),  issued  the  memorandum  Better  Buying 
Power:  Guidance  for  Obtaining  Greater  Efficiency  and  Productivity  in  Defense 
Spending  in  2010  to  target  affordability  and  control  cost  growth  (Carter,  2010a, 
2010b).  Subsequently  in  2012,  Under  Secretary  of  Defense  Frank  Kendall  launched 
the  Better  Buying  Power  2.0  initiative,  an  update  to  the  original  effort.  As  such, 
affordability  must  be  explicitly  considered  during  the  system  design  and  architecting 
phases.  These  memorandums  prescribe  high-level  guidelines  for  improving 
effectiveness  in  resource  usage  for  delivering  capabilities.  Guidelines  include 
mandating  affordability  as  a  requirement;  setting  an  affordability  target  as  a  Key 
Performance  Parameter  at  Milestone  A;  and  at  Milestone  B,  establishing  engineering 
trades  showing  how  each  key  design  feature  affects  target  cost. 

Focusing  on  Guideline  I,  affordability  has  now  become  a  design  requirement 
due  to  multiple  instances  of  failure  in  delivering  expected  technical  performance, 
increased  costs  and  schedule  delays  beyond  program  estimates,  and  altering  of 
requirements  during  program  execution  (Government  Accountability  Office  [GAO], 
2011, 2012,  2013).  With  explicit  considerations  of  affordability  early  in  and 
throughout  the  acquisition  process,  streamlining  decisions  is  imperative  to  achieving 
better  buying  power  (USD[AT&L]),  2013) 

Recent  Definitions  of  Affordability 

With  affordability  mandated  as  a  requirement,  it  has  become  increasingly 
prominent  within  the  DoD  and  defense  systems  community.  The  2010  Carter 
memorandum  (Carter,  2010a,  2010b)  defines  affordability  as  conducting  a  program 
at  a  cost  constrained  by  the  maximum  resources  the  Department  can  allocate  for 
that  capability.  The  International  Council  on  Systems  Engineering  (INCOSE)  defines 
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affordability  as  “the  balance  of  system  performance,  cost  and  schedule  constraints 
over  the  system  life  while  satisfying  mission  needs  in  concert  with  strategic 
investment  and  organizational  needs”  (INCOSE,  2011).  The  National  Defense 
Industrial  Association  (NDIA)  defines  affordability  as  “the  practice  of  ensuring 
program  success  through  the  balancing  of  system  performance  (KPPs),  total 
ownership  cost,  and  schedule  constraints  while  satisfying  mission  needs  in  concert 
with  long-range  investment,  and  force  structure  plans  of  the  DOD”  (NDIA,  2011). 

The  Defense  Acquisition  Guidebook  (DAG)  defines  affordability  as  “the  degree  to 
which  the  life-cycle  cost  of  an  acquisition  program  is  in  consonance  with  the  long- 
range  modernization,  force  structure  and  manpower  plans  of  the  individual  DoD 
Components,  as  well  as  for  the  Department  as  a  whole”  (DoD,  201 1 ,  Section  3.2). 

As  evidenced  by  this  set  of  definitions,  the  concept  of  affordability  not  only 
incorporates  cost,  but  also  schedule,  performance,  lifecycle,  and  all  of  these  things 
relative  to  a  larger  set  of  possible  investments.  An  affordable  system  is  cost  effective 
on  its  own,  and  relative  to  a  larger  system  investment  portfolio,  in  delivering  value  to 
the  customer  and  relevant  stakeholders.  Affordability  is  enhanced  if  the  system  is 
capable  of  satisfying  possible  changing  mission  requirements  over  the  system 
lifecycle.  Consequently,  a  system  developed  without  consideration  for  affordability  is 
one  that  has  been  designed  as  a  point  solution  in  isolation,  to  meet  a  specific  need 
at  a  specific  time,  possibly  requiring  the  procurement  of  an  entirely  new  system 
when  customer  needs  evolve  (Bobinis,  Haimowitz,  Tuttle,  &  Garrison,  2012).  While 
these  definitions  contain  the  vital  elements  that  constitute  affordable  systems,  they 
do  not  specify  how  the  cost  of  a  capability  should  be  measured  and  evaluated  (Tuttle 
&  Bobinis,  2012).  This  raises  the  question  of  how  affordable  solutions  can  be 
compared  against  one  another  during  Milestones  A  and  B  for  the  purpose  of  a  trade 
study  or  to  make  a  contract  award. 

Current  Ways  of  Understanding  Affordability 

One  way  of  framing  the  affordability  paradigm  currently  is  through  the 
Affordability  Triangle  shown  in  Figure  1  Conceptualized  by  Tuttle  and  Bobinis 
(2012),  it  depicts  the  relationship  among  capabilities,  performance,  schedule,  and 
budget.  The  triangle  shows  that  capabilities  form  the  baseline  of  any  acquisition 
process  and  that  it  is  important  to  first  establish  the  military  need  and  identify  how  it 
fits  within  the  existing  defense  portfolio.  This  process  is  conducted  early  in  the 
Materiel  Solutions  Analysis  Phase.  After  fixing  the  required  capability,  the 
affordability  decision  criteria  are  then  based  on  the  secondary  elements  of 
performance,  budget,  and  schedule.  These  elements  form  the  main  components  of 
affordability,  and  establishing  a  framework  based  on  them  ensures  its  compliance 
with  common  definitions.  The  balance  of  performance,  budget  and  schedule 
considerations  constitutes  a  standard  engineering  trade  study. 
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Figure  1.  The  Affordability  Triangle 

(Tuttle  &  Bobinis,  2012) 

To  transform  this  process  to  a  system  affordability  trade  study,  Tuttle  and 
Bobinis  (2012)  recommend  the  extension  of  the  system’s  time  horizon,  as  well  as 
the  inclusion  of  all  cost  elements  and  program  increments.  Based  on  current 
definitions  of  affordability  and  the  elements  contained  within  the  triangle,  they 
proposed  the  extraction  of  the  following  affordability  components: 

•  Required  Capabilities 

o  Identify  the  required  capabilities  and  the  time  phasing  for 
inclusion  of  the  capabilities 

•  Required  Capabilities  Performance 

o  Identify  and  specify  the  required  Measures  of  Effectiveness 
(MOEs)  for  each  of  the  capabilities 

o  Define  time  phasing  for  achieving  the  MOEs 

o  Identify  and  specify  Measures  of  Supportability  (MOSs) 

o  Define  time  phasing  for  achieving  the  MOSs 

•  Budget 

o  Identify  the  budget  elements  to  include  in  the  affordability 
evaluation. 

o  Time-phased  budget,  either  for  each  of  the  budget  elements,  or 
as  the  total  budget  (Tuttle  &  Bobinis,  2012,  p.7) 

Tuttle  and  Bobinis  illustrated  the  purpose  of  these  affordability  components 
through  an  example  depicted  in  Figure  2.  One  or  more  of  the  affordability  elements 
of  capabilities,  performance,  schedule,  or  budget  is  designated  as  the  decision 
criteria  that  will  be  used  to  perform  engineering  trade  studies  or  decision-making. 
The  remaining  affordability  elements  that  are  not  designated  as  decision  criteria  will 
become  specified  constraints. 
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Figure  2.  Determining  Affordable  Solution 

(Tuttle  &  Bobinis,  2012) 

In  this  notional  example,  capabilities  and  schedule  are  fixed  as  constants. 

This  results  in  a  relatively  straightforward  tradeoff  between  cost  and  performance,  as 
either  can  become  the  decision  criteria  while  the  other  becomes  the  constraint.  The 
maximum  budget  and  the  minimum  performance  thresholds  are  identified  and  they 
are  reflected  as  horizontal  and  vertical  lines  on  the  cost  and  performance  axes, 
respectively.  Design  solutions  below  the  maximum  budget  line  are  considered 
affordable  within  the  context  of  the  definition  in  the  2010  Carter  memorandum 
(Carter  2010a).  Solutions  to  the  right  of  the  minimum  performance  line  will  at  least 
satisfy  all  stakeholder  requirements,  and  they  are  considered  as  technically 
compliant  solutions.  The  green  rectangle  on  the  graph  is  formed  by  the  two 
threshold  lines  and  represents  the  region  containing  solutions  that  meet  the 
minimum  performance  requirements  and  are  within  the  maximum  budget. 

The  blue  curve  on  the  graph  is  another  notional  construct  that  connects  all 
solutions  that  have  the  “best  value.”  These  solutions  are  considered  Pareto  optimal, 
as  they  provide  the  best  possible  performance  achievable  given  the  minimum  cost.  If 
cost  is  set  as  the  decision  criteria,  the  “Low  Cost  Wins”  solution  will  be  selected 
because  it  requires  the  lowest  cost  expenditure  for  meeting  the  minimum 
performance  requirements.  However,  if  the  decision  criterion  is  performance,  the 
“Most  Bang  for  the  Budget”  solution  will  be  selected.  The  tradeoff  for  high 
performance  is  that  the  entire  budget  would  be  expended.  As  such,  affordable  and 
Pareto  optimal  solutions  in  this  notional  example  lie  along  the  blue  line  in  the  green 
rectangle. 

Through  this  example,  Tuttle  and  Bobinis  (2012)  demonstrate  that 
designating  the  main  affordability  elements  as  either  decision  criteria  or  constraints 
could  facilitate  the  identification  of  affordable  solutions  for  a  system  or  a  program. 
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They  also  suggest  that  system  or  program  affordability  trade  studies  can  be 
performed  more  accurately  if  the  budget  is  time  phased.  While  the  maximum  budget 
in  Figure  2  is  shown  as  a  single  number,  the  budget  can  actually  be  divided  and 
illustrated  as  an  annual  budget  for  a  sequence  of  fiscal  years  (Tuttle  &  Bobinis, 
2012).  With  a  time-phased  budget,  it  will  be  easier  for  system  architects  and 
stakeholders  to  identify  the  years  during  which  the  program  is  affordable  or 
unaffordable.  Similarly,  capabilities  and  performance  can  also  be  time  phased,  and 
their  requirements  for  a  particular  time  frame  may  not  be  met  in  the  context  of  an 
insufficient  budget. 

To  enable  the  conduct  of  system  or  program  affordability  trade  studies,  Tuttle 
and  Bobinis  also  analyze  the  lifecycle  cost  (LCC)  of  a  typical  program  and  break  it 
down  into  different  categories,  namely  research  development  testing  and  evaluation 
(RDT&E)  cost,  procurement  cost,  military  construction  (MILCON)  cost,  operations 
and  maintenance  cost,  and  finally,  military  personnel  (MILPERS)  cost.  With  the 
integration  of  time  phases,  the  cost  of  a  system  can  be  calculated  as  the  sum  of  all 
colors  of  money  across  all  time  increments.  The  equation  for  the  system  cost  can  be 
written  as  such: 


(1) 


where 

B^j  =  RDT&.E  Cost  of  Increment  j 

B2J  =  Procurement  Cost  of  Increment  j 

B3j  =  Milcon  Cost  of  Increment  j 

B4j  =  Operations  &  Maintenance  Cost  of  Increment  j 

BSJ  =  Military  Personnel  Cost  of  Increment  j 

i  =  1  or  5  for  a  single  budget  element  or  the  total  budget  respectively 

N  =  number  of  program  increments,  depending  on  the  number  of  time  phases 

This  is  similar  to  the  approach  formulated  by  Alter  (201 1 )  illustrated  in  Figure 
3,  which  aims  to  drive  affordability  considerations  by  breaking  down  total  ownership 
costs  (TOC)  into  their  constituent  costs.  This  is  followed  by  a  further  breakdown  to 
subsystem  cost  components,  plotted  on  a  graph  with  three  dimensions.  The 
subsystems  are  first  plotted  based  on  their  TOC  and  their  “ease  of  capture”  in  terms 
of  room  for  further  cost  reduction.  The  third  dimension  is  given  by  the  size  of  the 
point  denoting  each  component,  which  can  represent  a  “different  color  of  money,” 
such  as  operations  and  support  cost.  As  such,  areas  with  potential  for  cost 
reductions  can  be  identified  quickly,  facilitating  affordability  considerations  in  the 
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design  process.  An  understanding  of  the  cost  breakdown  also  helps  the  system 
architect  to  identify  areas  in  which  overhead  and  support  costs  can  be  reduced  in 
order  to  mitigate  overall  program  cost  growth.  This  principle  is  reinforced  by  the 
Carter  memorandum  (Carter,  2010a),  which  states  “the  ability  to  understand  and 
control  future  costs  from  a  program’s  inception  is  critical  to  achieving  affordability 
requirements.” 


Each  color  represents  a  different  cost  element  from  RDT&E  to  Disposal  — 


[ 


Build  up  Elements  into  TOC 

-  Acquisition - - O&S  - 


Break  down  TOC  to  System  Level 


- 

— 

_ 

|  Hull  Q  Propulsion  |f]  Electric  |||  NPES  Q  Auxiliary  f  Outfit  &  Furnishing  |  Armament 

A  represents  subsystem 
™  O&S  cost 


Cross-Function 
Team  Candidates 


0  (Difficult)  Ease  of  Capture  10  (Easy) 

Identity  and  gam  agreement  on  the  best  areas  tor  improvement 


Within  each  cost  category,  the  cost  is  split  into  systems  (and  lower) 


Figure  3.  Breakdown  of  TOC  Into  Subsystem  Costs  and  Identifying  Areas 

for  Improvement 

(Alter,  2011) 


Addressing  Affordability  as  a  Requirement 

Affordability  analysis  and  assessments  have  grown  in  importance  as  both 
government  bodies  and  industry  are  forced  to  consider  the  economics  of  system 
development  in  the  face  of  declining  defense  budgets.  A  multitude  of  quantitative 
methods  are  available  and  have  been  applied  to  several  affordability  case  studies 
within  the  defense  industry.  Some  of  these  methods  include  interval  cost  estimation 
and  system  lifecycle  analysis,  used  to  perform  affordability  analysis  on  a  number  of 
programs  owned  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA; 
Kroshl  &  Pandolfini,  2000). 

Traditionally,  cost  estimates  have  been  obtained  by  extrapolating  historical 
data  according  to  parameters  such  as  technical  complexity  or  concept  maturity. 
Such  estimates  are  often  flawed  and  far  from  actual  figures  because  they  fail  to 
capture  uncertainty.  Interval  cost  estimation  circumvents  this  problem  by  deriving 
cost  estimates  with  associated  probabilities;  this  process  is  illustrated  in  Figure  4. 
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Figure  4.  Interval  Cost  Estimation  Process  and  Its  Generated  Results 

(Kroshl  &  Pandolfini,  2000) 

Note.  Blue  indicates  the  main  path,  orange  indicates  optional  elements,  and  green  indicates 

data  sources. 


The  process  begins  by  breaking  down  the  total  cost  into  their  constituent 
costs  or  different  “colors  of  money”,  followed  by  developing  optimistic,  pessimistic, 
and  most-likely  cost  estimates  based  on  expert  opinion  or  other  relevant  parameters. 
Each  cost  element  is  then  associated  with  a  probability  distribution,  which  may  be 
modified  to  reflect  uncertainty  in  cost  estimates.  Finally,  all  constituent  costs  with 
their  assigned  probability  distributions  are  aggregated  to  obtain  summary  statistics 
and  a  probability  distribution  for  the  total  cost.  As  such,  interval  cost  estimation  helps 
to  build  up  a  stochastic  cost  model  that  captures  uncertainty  in  a  systematic  and 
traceable  manner.  Results  from  this  process  include  a  probability  curve  that 
illustrates  the  probability  of  a  cost  metric  meeting  or  exceeding  a  price  established  in 
the  affordability  requirement,  as  well  as  a  distribution  of  probabilistic  cost  estimates 
bounded  by  pessimistic  and  optimistic  thresholds.  This  helps  in  deriving  realistic  cost 
estimates  and  determining  whether  the  program  of  interest  remains  within  the 
affordability  requirement. 

System  lifecycle  analysis  is  much  like  the  process  described  by  Tuttle  and 
Bobinis  (2012),  in  which  cost  or  performance  can  be  set  as  the  decision  criteria  while 
the  other  is  left  as  the  constraint.  Metrics  used  in  this  analysis  are  drawn  from 
engineering  economics  and  financial  analysis,  and  they  include  net  present  value 
(NPV),  internal  rate  of  interest  (IRR),  and  learning  curve  functions.  The  calculation  of 
these  metrics  can  provide  decision-makers  with  useful  information  during  the  earliest 
studies  of  advanced  concepts. 
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Affordability  and  llities 

With  the  recent  emphasis  on  designing  for  affordability  as  a  requirement  in 
acquisition  management,  various  systems  engineering  approaches  have  been 
applied  to  design  systems  or  programs  that  are  more  manageable  under  explicit 
cost,  schedule,  and  performance  constraints.  However,  current  processes  for 
performing  early  lifecycle  affordability  tradeoffs  remain  underdeveloped.  In  recent 
strategies,  affordability  tradeoffs  have  been  limited  to  static  tradeoffs  of  systems  in 
current  operating  environments  or  in  single  point  futures.  Given  that  systems  exist  in 
a  dynamic  and  uncertain  world,  designing  for  affordability  not  only  necessitates  new 
methods  capable  of  evaluating  systems  across  many  possible  alternative  futures, 
but  also  a  new  philosophy  for  treating  the  affordability  paradigm. 

Since  the  application  of  ilities  (reliability,  flexibility,  robustness,  etc.)  in 
systems  engineering  can  potentially  lead  to  positive  results  in  the  design  process, 
affordability  may  be  treated  as  an  ility  that  can  drive  the  design  of  more  affordable 
yet  technically  sound  architectures.  With  affordability  as  an  ility,  advanced  systems 
engineering  methods  like  tradespace  exploration  may  be  applied  in  the  selection  and 
identification  of  affordable  designs  (Schaffner,  Wu,  Ross,  &  Rhodes,  2013). 

Tradespace  Exploration 

Tradespace  exploration  has  been  extensively  utilized  in  systems  design  and 
architecting  to  facilitate  the  identification  and  selection  of  architectures  that  best  fit 
stakeholder  requirements.  The  full  tradespace  of  a  system  is  spanned  by  all  its 
possible  design  alternatives,  and  it  represents  the  mapping  of  design  variables  to 
attribute  trades.  Enumerating  existing  design  variables  generates  all  possible 
alternatives,  and  expanding  the  tradespace  requires  the  generation  of  either  new 
design  variables  or  reconfigurations  of  existing  combinations  of  variables  (Ross, 
Hastings,  Warmkessel,  &  Diller,  2004).  Concurrently,  a  validated  model  for  the 
system  is  also  constructed  based  on  logical  assumptions  and  stakeholder 
preferences.  After  enumeration  and  evaluation,  the  tradespace  is  then  typically 
bounded  by  the  parameters  of  utility  and  cost  to  yield  the  solution  space.  The  Pareto 
frontier  of  these  design  points  is  then  established  and  traces  Pareto  subset  solutions 
with  the  best  tradeoffs  among  attributes  and  provides  the  highest  utility  for  a  fixed 
cost. 


The  process  of  tradespace  exploration  allows  comparison  of  many  designs  on 
a  common,  quantitative  basis,  and  it  structures  the  design  and  solution  space  to 
reflect  stakeholder  values.  As  thousands  of  designs  have  to  be  evaluated, 
tradespace  exploration  requires  computational  models  to  facilitate  the  assessment 
and  identification  of  “best”  or  “optimized”  solutions  along  the  Pareto  front,  thereby 
avoiding  fixation  on  local  point  solutions.  Uncertainties  in  cost,  schedule,  and 
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performance  are  also  accounted  for  in  the  model  through  a  variety  of  quantitative 
methods.  Beyond  establishing  the  Pareto  front,  tradespace  exploration  also  entails 
the  search  for  patterns  and  structures  emerging  within  dominated  regions  of  the 
tradespace  (Ross,  2006)  and  can  be  further  analyzed  in  terms  of  the  spatial 
distribution  of  points  or  clusters  within  the  solution  space.  Tradespaces  can  also 
become  dynamic  when  changes  in  utilities  and  costs  over  time  are  considered.  A 
sample  tradespace  is  shown  in  Figure  5  (Ross,  2006),  which  also  places  the  various 
types  of  trades  into  perspective:  (1)  point  solution,  (2)  Pareto  subset,  (3)  Pareto 
frontier,  (4)  full  tradespace,  and  (5)  dynamic  tradespace. 


Figure  5.  Tradespace  Depicting  Different  Trade  Types 

(Ross,  2006) 

The  tradespace  exploration  paradigm  can  facilitate  the  system  design 
generation,  evaluation,  and  selection  process,  and  utilization  of  resources  to  meet 
evolving  stakeholder  requirements.  Tradespace  exploration  has  already  been 
applied  to  select  best  designs  for  numerous  space  systems  and  even  best  options 
for  policy  robustness.  As  such,  dynamic  system  tradespaces  with  consideration  of 
temporality  may  be  used  to  perform  affordability  tradeoffs  early  in  the  lifecycle  and 
demonstrate  changes  in  costs  as  major  decision  parameters  and  time  to  completion 
are  varied. 

Decision-making  frameworks  are  often  applied  within  the  tradespace 
exploration  process  to  expedite  the  identification  and  selection  of  “best”  or 
“optimized”  designs.  Multi-Attribute  Utility  Theory  (MAUT)  is  one  such  framework 
within  decision  analytics  that  may  be  used  to  perform  affordability  tradeoffs  as  it 
allows  the  capturing  of  stakeholder  preferences  for  simultaneous  multiple  objectives 
(Keeney  &  Raiffa,  1993).  It  uses  mathematical  representations  to  capture  complex 
tradeoffs  and  interactions  among  the  attributes  that  have  been  translated  from 
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articulated  design  and  risk  preferences  (Ross,  2003).  Multi-Attribute  Tradespace 
Exploration  (MATE)  is  a  conceptual  design  method  that  applies  MAUT  to  model  and 
simulation-based  design  (Ross  et  al.,  2004). 

MATE  resolves  conflicting  and  subjective  evaluations  of  decision-making 
processes  by  combining  various  single-attribute  utility  functions  for  every  attribute  of 
interest  into  a  single  function  that  quantifies  how  a  decision-maker  values  different 
attributes  relative  to  one  another.  The  MATE  process  maps  the  design  and  attribute 
space  of  the  system  to  the  solution  space,  which  is  typically  defined  by  the 
parameters  of  aggregated  utility  and  cost.  The  MATE  process,  illustrated  in  Figure  6, 
culminates  in  the  generation  of  a  tradespace  that  allows  thousands  of  design 
alternatives  to  be  compared  on  a  common,  quantitative  basis.  As  such,  MATE 
provides  decision-makers  a  prescriptive  framework  for  selecting  preferred  designs 
that  can  be  carried  forward  for  more  detailed  sensitivity  analysis  and  eventually 
development. 


Figure  6.  Multi-Attribute  Tradespace  Exploration  Framework 

(Ross,  2006) 


MATE  provides  a  method  for  identifying  affordable  systems,  forcing  alignment 
of  solutions  to  needs,  facilitating  cross-domain  socio-technical  conversation,  and 
discovering  compromise  solutions.  By  explicitly  considering  cost  and  schedule 
constraints,  affordability  tradeoffs  may  be  performed  through  a  complete  exploration 
of  the  tradespace,  and  affordable  solutions  may  be  identified  in  the  solution  space. 

Epoch-Era  Analysis 

Epoch-Era  Analysis  (EEA)  is  a  computational  scenario  planning  approach 
(Roberts,  Richards,  Ross,  Rhodes,  &  Hastings,  2009;  Ross  &  Rhodes,  2008)  that 
can  be  used  for  the  analysis  of  systems  operating  in  dynamic  environments.  EEA 
considers  changes  in  needs  and  context  in  tradeoffs,  along  with  the  system  itself. 
One  of  the  advantages  of  EEA  is  that  it  allows  for  consideration  of  temporality  and 
exogenous  impacts  in  the  analysis  of  system  capabilities  for  cost.  Often  shifts  in 
contexts  occur  more  frequently  than  system  development  timelines,  such  as 
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changes  in  budgets,  administrations,  and  warfighter  needs.  Figure  7  illustrates  this 
mismatch  in  terms  of  development  phase,  possible  transitions  in  the  system,  and 
“epoch”  time  periods  of  fixed  context  (includes  technologies  and  policies)  and  needs. 
During  the  lifecycle,  there  are  various  system  changes  (i.e.,  “transitions”)  that  may 
occur,  resulting  in  fluctuations  that  feed  back  into  the  program. 


Now 

2030 

Era  j" 

1 

Epoch  |  2-8  years  1 1  2-8  years 

2-8  years  ||  2-8  years 

2-8  years  ||  2-8  years 

System  |  Concept  Development  (3  yrs)  | 

Design.  Build.  Test.  Launch  (5  yrs) 

Fly  &  Possible  Upgrades 

Transitions 

Tech  Insertion  Opportunities 

More  Vehicles 

Design  Modifications 

Software  Infrastructure  Changes 

Changing  Interaction  with  (changing)  Environment 

Figure  7.  System  Lifecycle  Phases,  Transitions,  Epochs,  and  Eras 


To  effectively  evaluate  the  impact  of  dynamic  variation  in  costs  with  tradeoffs 
in  decision  parameters  and  time  to  completion,  EEA  may  be  applied  to  enable 
designing  for  affordability.  EEA  has  been  developed  to  consider  and  clarify  the 
effects  of  changing  contexts  and  needs  over  time  on  the  perceived  value  of  a  system 
in  a  structured  manner  (Ross,  2006;  Ross  &  Rhodes,  2008).  Instead  of  discretizing 
the  system  lifecycle  according  to  traditional  system  milestones,  EEA  discretizes  the 
lifecycle  according  to  impactful  changes  in  the  operating  environment,  stakeholders, 
or  the  system  itself,  through  the  constructs  of  epochs  and  eras. 

An  epoch  is  a  time  period  of  fixed  contexts  and  needs  under  which  the 
system  operates,  and  it  can  be  characterized  using  a  set  of  variables  that  define  any 
factor,  such  as  technology  level  and  supply  availability,  which  impacts  the  usage  and 
value  of  the  system.  An  ordered  sequence  of  epochs  constitutes  an  era  and 
describes  the  potential  progression  of  contexts  and  needs  over  time.  Any  futures 
relevant  to  system  performance  or  costs  can  be  described  through  assignments  to 
the  available  epoch  variables,  providing  a  form  of  computational  scenario  planning. 

Epoch  variables  define  the  context  and  time  that  the  system  may  be  operating 
in.  They  can  affect  the  usage  and  value  of  the  system  and  may  be  in  the  form  of 
weather  patterns,  political  scenarios,  financial  situations,  operational  plans,  and 
availability  of  other  technologies.  Appropriate  epoch  variables  for  an  analysis  will 
include  exogenous  uncertainty  factors  that  will  affect  the  perceived  success  of  the 
system  (Ross,  2006).  Figure  8  shows  the  temporal  progression  of  a  system  as 
needs  and  contexts  change.  The  vertical  columns  represent  the  epochs  that  are 
time-ordered  to  form  an  era.  The  different  colors  of  these  epochs  represent  changes 
in  context.  The  horizontal  bands  capturing  the  minimum  and  desired  expectation 
levels  for  that  epoch  represent  expectations  (needs).  Contexts  and  needs  can 
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change  independently  of  one  another  as  shown  by  the  different  horizontal  bands. 
The  system  can  potentially  exhibit  a  different  ility  in  each  context. 
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Figure  8.  System  Needs  Versus  Expectations  Across  Epochs  of  the  System 

Era 

(Ross,  2006) 


Initially  in  Epoch  1,  the  system  exceeds  the  needs  of  stakeholders.  Later  in 
Epoch  2  and  Epoch  3,  the  context  and  needs  change  respectively.  In  these  cases, 
the  system  still  meets  expectations  and  displays  value  robustness.  The  shift  to 
Epoch  4  represents  a  new  need  of  the  system,  which  the  system  satisfies,  displaying 
versatility.  In  Epoch  4,  the  system  does  not  exceed  all  the  needs,  but  does  meet  the 
minimum  required  level  and  remains  successful.  Finally,  the  shift  to  Epoch  5 
represents  the  need  for  a  changeable  system.  As  seen,  the  system  is  not  robust  or 
versatile  to  new  needs  and  context,  and  requires  change  in  order  to  remain 
successful  (Beesemyer,  Ross,  &  Rhodes,  2012). 

As  demonstrated  by  this  notional  system  trajectory,  EEA  can  structure 
consideration  of  changing  contexts  and  needs  on  system  success,  and  suggest 
strategies  for  how  to  sustain  value  in  both  the  short  run  and  the  long  run.  As  such, 
EEA  may  be  used  in  conjunction  with  MATE  to  help  in  the  search  for  designs  that 
are  affordable  across  many  discrete  epochs  and  design  for  ilities  in  systems  (Ross, 
2006).  This  approach  provides  an  intuitive  base  upon  which  to  perform  analysis  of 
value  delivery  over  time  for  systems  under  the  effects  of  changing  circumstances 
and  operating  conditions,  an  important  step  to  take  when  evaluating  large-scale 
engineering  systems  with  long  lifespans. 

Figure  9  illustrates  how  alternative  eras  can  be  constructed  from  the 
enumerated  epochs  in  time-ordered  sequence.  Eras  can  be  constructed  either  by 
selecting  pre-defined,  “hand-picked”  epochs  to  fit  imagined  future  scenarios  of 
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interest,  or  eras  can  be  constructed  in  a  combinatorial,  logic-based  automated 
fashion.  Automated  era  construction  can  be  random,  constrained  by  a  given  set  of 
conditions,  or  can  be  iterative,  so  that  the  selection  of  each  subsequent  epoch 
depends  on  the  preceding  one.  The  sequential  ordering  of  selected  epochs 
produces  an  emergent  path  dependence  of  value  over  time  for  each  system 
alternative  (i.e.,  the  optimal  design  strategy  given  an  uncertain  future  may  depend 
on  the  order  of  future  events). 


Figure  9.  Alternative  Eras  Constructed  From  Enumerated  Epochs 

(Rader  et  al.,  2010) 

Epoch-Era  Analysis  is  first  used  to  gather  performance  and  uncertainty 
information  of  selected  alternative  system  concepts  operating  through  different 
epochs  and  eras.  This  is  accomplished  through  enumeration  of  possible  epochs 
(exogenous  uncertainties),  basic  performance  models  (to  evaluate  the  performance 
of  alternative  concepts  within  particular  epochs),  and  related  analyses  (“multi-epoch 
analysis”  and  “era-level  analysis”). 

Figure  10  shows  the  enumeration  of  epochs  for  an  example  of  a  U.S.  Coast 
Guard  (USCG)  Offshore  Patrol  Cutter  program,  in  which  five  selected  epoch 
categories  have  one  or  more  associated  descriptors  (or  variables),  each  having  units 
and  ranges.  Constraints  are  also  noted. 
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Epoch  Descriptor 
Category 

Epoch  Descriptor 

Units 

Range 

Constraints 

Technology 

Availability  of  VUAV 
Technology 

Level 

Small-Large 

Requires  hangar  storage 

C4ISR  Racks 

Level 

Small-Large 

Original  design  space,  weight,  and  power 

Small  Boat  Size 

ft 

24-35 

C4ISR  Info  to/from  cutter  remain  same 

Policy 

Engine  Emmissions  Rating 

Tier 

2  to  4 

Weight 

Discharge  Copper  Content 

Level 

Low-Medium-High 

Maintain  original  system  service  life 

SCI  F  Size 

Level 

Low-Medium-High 

Location  near  operational  spaces 

Budget 

Project  Baseline 

% 

-20 

Systems  of  Systems 

Operational  Availability 

Dimensionless 

0.85-0.92 

Major  equipment  remains  same 

Range  Increase 

% 

5  to  20 

Same  operational  conditions 

Helicopter  Weight  Increase 

% 

5  to  50 

Size  less  than  HH-60 

Missions 

Ice  Region  Use 

Level 

Low-Medium-High 

Floating  ice  capability  only 

Equipment  Storage 

ft3 

Small-Large 

Storage  only 

Water/Food  Storage 

%  Increase 

5  to  20 

Same  operational  conditions 

Figure  10.  Enumeration  of  Epochs  Example  for  Coast  Guard  Offshore  Patrol 

Cutter 

(Schofield,  2010) 

The  two  types  of  analyses  within  EEA  each  provide  unique  insights  into  the 
impact  of  uncertainty  and  dynamics  on  the  value  (capabilities  at  cost)  produced  by  a 
system.  Multi-Epoch  Analysis  is  an  approach  for  identifying  systems  robust  to 
exogenous  uncertainties  involving  contexts  and  needs  (Fitzgerald  &  Ross,  2012). 
This  is  accomplished  through  enumeration  of  possible  epochs  that  might  be 
encountered  by  a  system.  These  epochs  encapsulate  a  particular  “point  sampling”  of 
uncertain  futures.  Analysis  can  be  done  looking  across  all  of  these  point  futures  to 
identify  alternatives  that  perform  well  (i.e.,  deliver  high  utility  at  low  cost),  that  is, 
alternatives  that  are  robust  to  changes  in  contexts  and  needs  (Ross,  Rhodes,  & 
Hastings,  2009).  This  type  of  analysis  calculates  metrics,  such  as  Normalized  Pareto 
Trace  (NPT),  which  represents  the  fraction  of  epochs  in  which  a  given  design 
alternative  is  considered  cost-utility  efficient,  as  shown  in  Figure  1 1 . 
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Epoch 


Figure  11.  Epochs  as  Alternative  "Point"  Futures  (I)  and  Multi-Epoch 

Analysis  (r) 

Multi-Attribute  Expense  (MAE) 

Designing  for  affordability  is  not  only  concerned  with  the  monetary  lifecycle 
cost  of  a  system,  but  also  its  “schedule”  of  development  and  its  responsiveness  to 
emerging  needs  (INCOSE,  2011).  However,  such  temporal  considerations  are  often 
difficult  or  impossible  to  represent  in  dollars  and  these  different  colors  of  money  may 
also  be  spent  with  differing  degrees  of  ease.  This  is  not  easily  captured  by  general 
utility.  A  possible  solution  to  affordability  analysis  is  the  use  of  the  Multi-Attribute 
Expense  (MAE)  function  (Diller,  2002),  which  aggregates  these  different  types  of 
dollar  budgets.  MAE  is  formulated  similarly  to  a  MAU  function  (Keeney  &  Raiffa, 
1993),  with  the  utility  function  replaced  by  an  expense  function  E(X). 

N 

KE(X)  +  1  =  PJlIfilcEM)  +  1) 

'“l  (2) 


Expense  refers  to  system  aspects  that  the  designer  wants  to  keep  at  low 
levels,  a  concept  akin  to  the  notion  of  negative  utility.  Expense  is  focused  on  “what 
goes  into  a  system”  in  contrast  to  utility,  which  is  focused  on  “what  comes  out  of  a 
system.”  Quantified  on  a  0  to  1  scale,  an  expense  level  of  one  denotes  complete 
dissatisfaction,  and  an  expense  level  of  zero  denotes  minimal  dissatisfaction.  As 
such,  a  stakeholder  typically  demands  maximal  utility  and  minimal  expense  in  an 
ideal  design. 

Like  MAU,  an  MAE  function  requires  careful  construction  through  stakeholder 
interviews  to  elicit  informed  responses  and  aggregate  preferences  to  capture 
articulated  value.  As  MAE  is  a  dimensionless,  non-ratio  scale  metric,  an  entity  with 
twice  the  MAE  number  over  another  does  not  imply  that  it  is  twice  as  expensive  in 
terms  of  monetary  value.  Since  temporal  elements  like  schedule  constraints  and 
time-to-build  have  extensive  leverage  on  the  different  colors  of  money,  the  MAE  can 
be  extended  to  affordability  applications  in  federal  acquisition  processes.  Instead  of 
comparing  monetary  costs  against  utility,  EEA  and  MATE  may  be  modified  to 
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compare  MAE  against  MAU  in  order  to  perform  an  affordability-driven  analysis  that 
captures  the  elements  of  both  time  and  costs. 

Therefore,  MATE,  EEA,  and  MAE  can  be  combined  to  yield  an  enabling 
method  for  making  effective  comparisons  of  benefits  and  costs  across  a  range  of 
alternative  futures  (Schaffner  et  al.,  2013).  By  explicitly  accounting  for  cost, 
schedule,  and  performance  requirements  over  time,  the  method  is  able  to  account 
for  system  changes  due  to  shifts  and  perturbations,  manage  lifecycle  differences 
between  subsystem  components,  evaluate  feedback,  and  be  adaptive  to  evolving 
system  behaviors.  As  affordability  is  a  concept  evaluated  over  time,  such  a  method 
can  provide  structured  options  for  improvement  to  enable  enhanced  design  for 
affordability. 

METHOD  FOR  AFFORDABILITY  TRADEOFFS  UNDER 
UNCERTAINTY 

The  outcome  of  this  research  is  a  nine-process  method,  applied  in  the 
selected  case,  which  extends  from  prior  research  at  the  Massachusetts  Institute  of 
Technology  (MIT).  The  purpose  of  the  method  is  to  perform  affordability  tradeoffs 
under  uncertainty.  The  overall  structure  of  the  proposed  method  consists  of  nine 
processes,  which  are  grouped  into  three  distinct  parts:  information  gathering 
(Processes  1  through  3),  alternatives  evaluation  (Process  4),  and  alternatives 
analysis  (Processes  5  through  9).  A  graphical  representation  of  the  method  is  shown 
in  Figure  12. 

The  information-gathering  portion,  Processes  1  through  3,  consists  of  defining 
the  context  and  problem  statement,  stakeholders  and  respective  needs,  and 
contextual  variables.  The  alternatives  analysis  portion,  Processes  5  through  9, 
compares  the  dynamic  properties  of  potential  designs  across  the  potential  futures 
that  the  system  may  encounter.  These  two  main  portions  of  the  proposed  method 
are  bridged  by  Process  4  (Design-Epoch  Tradespaces  Evaluation),  which  can 
provide  feedback  to  decision-makers  and  stakeholders,  creating  an  opportunity  to 
revisit  the  information  gathering  processes.  Process  4  also  provides  a  cursory 
analysis  of  potential  designs  in  preparation  for  the  more  in-depth  alternatives 
analysis  in  the  second  half  of  the  method. 
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Information  Gathering 


I 


1.  Value-Driving 

2.  Value-Driven 

3. Epoch 

Context  Definition 

Design  Formulation 

Characterization 

Alternatives  Evaluation 


4.  Design-Epoch 
Tradespaces  Evaluation 


Alternatives  Analysis 


5.  Single  Epoch 

6.  Multi-Epoch 

7.  Era 

8.  Single-Era 

9.  Multi-Era 

Analyses 

Analysis 

Construction 

Analyses 

Analysis 

Figure  12.  A  Graphical  Overview  of  the  Gather-Evaluate-Analyze  Structure  of 

the  Method 

The  processes  of  the  proposed  method,  with  brief  descriptions  of  the  activities 
involved,  are  as  follows: 

Process  1:  Value-Driving  Context  Definition 

The  first  process  of  the  proposed  method  involves  development  of  the  basic 
problem  statement.  The  stakeholders  are  identified,  relevant  exogenous 
uncertainties  are  elicited,  and  an  initial  value  proposition  is  formed.  The  resources 
available  to  each  stakeholder  are  examined  along  with  the  associated  uncertainties. 

Process  2:  Value-Driven  Design  Formulation 

The  second  process  begins  by  defining  the  needs  statements  for  all 
stakeholders,  which  become  the  attributes  of  system  performance,  along  with  utility 
functions  describing  each  stakeholder’s  preference  for  each  attribute.  The 
stakeholder  resources  statements  are  also  elicited  (with  corresponding  expense 
functions),  which  then  become  the  attributes  of  the  system’s  expense  function.  The 
system  solution  concepts  are  proposed  from  past  concepts  or  expert  opinions. 

These  concepts  are  decomposed  into  design  variables  of  the  system. 

Process  3:  Epoch  Characterization 

In  this  process,  the  key  contextual  uncertainties  are  parameterized  as  epoch 
variables,  and  possible  future  contexts  are  identified.  Uncertainties  in  stakeholder 
needs  are  elicited.  Uncertainties  in  resource  supply  and  availability  are  also 
identified,  along  with  changes  to  stakeholder  preferences  on  resource  usage. 
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Process  4:  Design-Epoch  Tradespaces  Evaluation 

This  process  utilizes  modeling  and  simulation  to  map  the  design  and  epoch 
variables  to  system  performance  attributes  and  expense  attributes.  Stakeholders’ 
utility  and  expense  functions  are  then  used  to  generate  the  MAU  and  the  MAE  for 
each  design,  within  each  epoch. 

Process  5:  Single  Epoch  Analyses 

This  process  includes  the  analysis  of  MAU  and  MAE  of  alternatives  within 
particular  epochs,  including  designs  graphically  compared  on  an  MAU  versus  MAE 
scatterplot  for  any  given  epoch  (time  period  of  fixed  operating  context  and 
stakeholder  needs).  Within-epoch  metrics,  such  as  yield,  give  an  indication  of  the 
difficulty  of  a  particular  context  and  needs  set  for  considered  designs. 

Process  6:  Multi-Epoch  Analysis 

After  completing  the  traditional  tradespace  exploration  activities  of  Process  5, 
in  which  the  practitioner  compares  potential  designs  within  a  particular  epoch, 
metrics  are  derived  from  measuring  design  properties  across  multiple  (or  all)  epochs 
to  give  insight  into  the  impact  of  uncertainties  on  potential  designs,  including 
evaluation  of  short-run  passive  and  active  strategies  for  affordability  (i.e. ,  efficient 
MAU  at  MAE).  In  addition,  resource  usage  can  be  analyzed  to  identify  designs  that 
are  robust  to  the  factors  identified  in  Process  3  (e.g.,  decreasing  budgets  or  labor 
availability). 

Process  7:  Era  Construction 

This  process  constructs  multiple  sequences  of  various  fixed  duration  epochs 
to  create  alternative  eras,  which  are  long-term  descriptions  of  possible  futures  for  the 
system,  its  context,  and  stakeholder  needs.  This  process  can  be  performed  with  the 
aid  of  expert  opinion,  probabilistic  models  (e.g.,  Monte  Carlo  or  Markov  models),  and 
scenarios  of  interest  to  stakeholders. 

Process  8:  Single-Era  Analyses 

This  process  examines  the  time-dependent  effects  of  an  unfolding  sequence 
of  future  epochs  (era)  created  in  Process  7.  By  examining  a  particular  series  of 
epochs  for  a  given  length  of  time,  decision-makers  can  identify  potential  strengths 
and  weaknesses  of  a  design  and  better  understand  the  potential  impact  of  path- 
dependent,  long-run  strategies  for  affordability. 
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Process  9:  Multi-Era  Analysis1 

This  process  extends  Process  8  by  evaluating  the  dynamic  properties  of  a 
system  across  many  possible  future  eras,  identifying  patterns  of  strategies  that 
enable  affordability  across  uncertain  long-run  scenarios. 

CASE  APPLICATION  AND  ANALYSIS 

Introduction  and  Background 

The  analysis  of  system  affordability  covers  many  of  the  aspects  previously 
discussed,  including  the  system  development  schedule,  various  types  of  expenses, 
and  the  level  of  those  expenses  in  dynamic  operating  environments  over  the  system 
lifecycle.  Ideally,  these  factors  must  be  balanced  with  the  value  delivery  of  the 
system,  since  a  system  providing  minimal  performance  might  be  affordable  but  not 
desirable.  In  this  case  study,  an  established  method  for  conceptual  system  design  is 
used  to  consider  all  of  these  factors,  augmented  by  several  metrics  introduced  to 
cover  resource-centric  concerns.  The  first  of  these  measures  is  the  Multi-Attribute 
Expense  (MAE)  function,  intended  to  aggregate  stakeholder  preferences  on 
resource  consumption.  The  second  is  the  Max  Expense  metric,  which  gives  the 
maximum  resource  expenditure  for  any  resource  across  time.  The  third  is  the 
Expense  Variability  metric,  which  reflects  the  stability  of  resource  consumption  over 
time.  The  incorporation  of  all  of  these  metrics  enables  the  design  method  to  directly 
inform  analysts  and  decision-makers  of  the  relative  affordability  of  each  potential 
design  under  consideration,  whether  relative  to  one  another  or  to  established 
projected  budget  levels. 

The  method  developed  in  this  research  is  based  upon  the  Responsive 
Systems  Comparison  (RSC)  method  (Ross  et  al.,  2008).  The  RSC  method  was 
developed  to  aid  in  the  design  of  complex  systems  across  many  domains,  allowing 
effective  anticipation  of  future  contexts  and  needs  relevant  to  system  design  choices 
early  in  the  lifecycle  through  the  Epoch  Era  Analysis  (EEA)  approach  (Ross  & 
Rhodes,  2008).  RSC  has  been  applied  to  several  case  applications  ranging  from 
satellite  systems  (Ross,  Rhodes,  McManus,  Hastings  &  Long,  2009)  to  a  proposed 
Coast  Guard  replacement  vessel  (Schofield,  2010).  The  method  applied  in  the 
present  study  is  the  earlier  described  nine-process  method,  focused  on  affordability 
analysis,  as  applied  to  a  hypothetical  Next-Generation  Combat  Ship  (NGCS).  NGCS 
is  conceived  as  a  larger  version  of  the  Navy’s  current  Littoral  Combat  Ship  (LCS) 
that  would  support  air  and  sea  operations  over  diverse  areas  of  interest  for  the  next 


The  process  is  not  demonstrated  by  the  current  study  due  to  the  representative  nature  of  the 
analysis  but  is  described  here  for  completeness. 
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30  years.  Schofield’s  (2010)  application  of  the  original  RSC  method  was  to  a  smaller 
naval  application,  the  Coast  Guard’s  Offshore  Patrol  Cutter  (OPC).  The  current  case 
draws  from  the  design  variables,  attributes,  and  epochs  from  Schofield’s  OPC  study, 
augmented  by  evaluation  of  outputs  using  the  MIT  Math  Model.  The  MIT  Math  Model 
is  a  standard  naval  modeling  tool  regularly  used  by  the  MIT  Ocean  Engineering 
group  (formerly  department)  for  the  evaluation  of  potential  designs  for  naval  frigates 
(slightly  larger  than  the  LCS). 

The  proposed  NGCS  requirements,  therefore,  reflect  some  similarity  with  both 
the  OPC  and  LCS.  For  example,  the  OPC  is  designed  to  operate  in  a  variety  of 
mission  areas,  including  ports,  near  shore,  and  open  sea,  with  a  range  in  excess  of 
8,500  nautical  miles  and  endurance  minimum  of  45  days  (Schofield,  2010).  The  LCS 
is  designed  to  have  a  range  in  excess  of  3,500  nautical  miles  and  an  endurance  of 
21  days.  The  NGCS  that  is  the  focus  of  this  study,  meanwhile,  is  required  to  operate 
in  mission  areas  at  least  as  varied  as  the  OPC,  have  a  minimum  endurance  of  30 
days,  and  have  a  range  in  excess  of  4,000  nautical  miles.  It  is  anticipated  that  the 
operating  context  of  the  NGCS  is  largely  similar  to  that  of  the  OPC,  so  many  of  the 
NGCS’s  contextual  variables  mirror  those  from  the  OPC  study. 

Application  of  the  Proposed  Method  to  the  NGCS 

While  the  OPC  and  LCS  systems  are  designed  for  many-unit  acquisitions  occurring 
over  a  period  of  several  years,  the  present  study  examines  only  a  single  unit 
acquisition  for  the  purposes  of  demonstrating  the  salient  points  of  analysis. 

Process  1 :  Value-Driving  Context  Definition 

The  value-driving  context  for  the  OPC  is  made  up  of  the  value  propositions  as 
well  as  the  key  stakeholders  involved  in  decision-making  and  funding.  Schofield 
(2010,  p.  82)  defines  the  value  propositions  for  each  stakeholder  as  follows: 

Project  Office:  Provide  a  new  cutter  fleet  that  meets  operational 
requirements  within  a  defined  budget  level  and  whose  delivery  coincides  with 
the  decommissioning  of  the  current  WMEC  fleet. 

Sponsor:  Develop  operational  requirements  that  meet  the  mission  needs  of 
the  Coast  Guard  and  Coast  Guard  user  requirements. 

Technical  Authorities:  Ensure  that  the  new  developed  system  meets  legacy, 
external  constraints,  and  design  standards  with  technologies  that  maximize 
capability  within  established  risk  requirements. 

It  is  clear  from  the  value  propositions  that  concern  for  resource  usage  is  not 
consistent  across  stakeholders;  as  one  might  expect,  each  stakeholder  has  different 
expectations  and  goals  with  regard  to  resources  involved  in  the  project.  The  project 
office  specifically  addresses  two  standard  resources:  budget  (“defined  budget  level”) 

ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  -  21  - 

NAVAL  POSTGRADUATE  SCHOOL 


and  schedule  (“delivery  to  coincide  with  The  sponsor  appears  to  be  primarily 
concerned  with  the  mission  needs  and  user  requirements  of  the  organization,  and 
resource  usage  is  not  of  primary  concern.  The  technical  authorities’  value  statement 
includes  the  aspect  of  technological  resources  that  enable  core  capabilities. 

Because  each  of  the  stakeholders’  value  propositions  reveal  the  different 
priorities  of  their  respective  organizations,  the  present  case  of  the  NGCS  combines 
the  various  points  of  view  into  one  representative  stakeholder  for  simplicity  of 
analysis.  This  stakeholder  desires  to  provide  the  new  fleet  of  USN  frigates  for  use  in 
air  and  sea  operations  in  a  variety  of  operating  areas.  In  the  second  process  of  the 
proposed  method,  interviews  with  this  stakeholder  will  better  reveal  the  relevant 
preferences  on  the  usage  of  the  resources. 

Process  2:  Value-Driven  Design  Formulation 

The  second  process  builds  upon  the  initial  system  context  definition  by  first 
proposing  the  system  design  concept  and  then  eliciting  the  attributes  desired  by  (as 
well  as  expense  attributes  of  importance  to)  the  stakeholder.  Through  stakeholder 
interviews,  the  attributes’  characteristics  can  be  determined  and  weighted  according 
to  the  preferences  revealed.  In  this  case,  the  weights  placed  on  each  attribute  reflect 
the  “combined”  stakeholder  from  Process  1 .  Two  types  of  attributes  are  delineated  in 
the  results:  those  attributes  which  represent  resources  that  would  ideally  be 
conserved  from  the  stakeholder’s  perspective,  or  “expense”  attributes  (e.g., 
acquisition  cost,  crew  size),  and  those  attributes  which  represent  performance  that 
would  ideally  be  maximized,  or  “utility”  attributes  (e.g.,  range,  speed).  The  results  of 
this  activity  can  be  seen  in  Figure  13. 


Mission  Statement: 

Provide  new  frigate  fleet  for  the  USN. 

Objective: 

Provide  defense,  air,  and  sea  support 
across  the  defined  mission  areas. 

Attribute  (Acronym) 

Units 

Range 

["worst"  to  "best"] 

Weight 

(.lt°l) 

Acquisition  Cost  (Acq.) 

$  million 

1500 

500 

0.4 

Lifecycle  Cost  (LC) 

$  million 

7500 

3000 

0.3 

Initial  Operating  Cap.  (IOC) 

year 

2020 

2018 

0.2 

Crew  Size  (Crew) 

#  persons 

250 

200 

0.1 

Displacement  (Disp.) 

cubic  ft. 

11000 

4000 

0.2 

Range 

nm 

4500 

10000 

0.2 

Speed 

kts 

12 

25 

0.1 

Air  Capability  (Air  Cap) 

hrs/day 

2 

12 

0.2 

Endurance  (Endr) 

days 

30 

60 

0.1 

Small  Boat  Capability  (SBCap) 

#ppl-hrs/day 

20 

100 

0.2 

Figure  13.  Decomposition  of  Mission  Statement  Into  Attributes 

The  design  concepts  are  then  partitioned  into  potential  design  variables  for 
the  proposed  system.  To  better  identify  the  key  design  drivers,  the  relationships  of 
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design  variables  to  utility  attributes  and  expense  attributes  are  then  assessed 
qualitatively  by  the  values  none,  low,  medium,  or  high  impact,  using  a  Design-Value 
Matrix  (DVM)  with  values  of  0,  1 ,  3,  and  9  as  a  visual  aid  for  this  activity.  Following 
Schofield’s  (2010)  example  of  decomposing  the  value  propositions  generated  in 
Process  1  to  infer  the  utility  attributes,  the  present  study  creates  a  DVM  mapping  the 
impact  of  design  variables  to  the  resource  expenditures  of  the  system.  The  impacts 
are  assessed  of  each  design  variable  on  each  expense  attribute  in  addition  to  the 
utility  attributes,  generating  the  DVM  shown  in  Figure  14. 


Design-Value  Mapping  (Notional  Values) 

Representative  Stakeholder 

Expense 

Attributes 

Design  Variables 

Length 

Beam 

Draft 

Prop  Type 

Hull  Material 

Deckhouse  Material 

Defense  Cabability 

Helos 

ASUW 

AAW 

Total  Impact 

Acquisition  Cost 

9 

3 

3 

3 

3 

3 

3 

3 

3 

3 

36 

Lifecycle  Cost 
IOC 

9 

3 

3 

9 

1 

1 

1 

1 

1 

1 

30 

1 

3 

1 

3 

1 

0 

3 

1 

1 

1 

15 

Crew  Size 

3 

1 

1 

1 

1 

1 

3 

3 

3 

3 

20 

Total 

22 

10 

8 

16 

6 

5 

10 

8 

8 

8 

Utility 

Attributes 

Displacement 

Range 

9 

9 

3 

3 

9 

3 

3 

3 

1 

1 

44 

9 

3 

1 

9 

3 

1 

1 

3 

1 

1 

32 

Speed 

9 

3 

3 

9 

3 

1 

3 

1 

1 

1 

34 

Air  Capacity 
Endurance 
Small  Boat  Cap. 

3 

3 

1 

0 

1 

1 

3 

9 

3 

3 

27 

1 

3 

1 

3 

1 

1 

3 

3 

3 

3 

22 

3 

3 

1 

0 

1 

1 

3 

3 

3 

3 

21 

Total 

34 

24 

10 

24 

18 

8 

16 

22 

12 

12 

Figure  14.  A  Design-Value  Matrix  Reflecting  the  Notional  Impact  of  Design 

Variables  on  Attributes 

Several  benefits  exist  from  creating  such  a  DVM.  First,  by  summing  the  rows 
and  columns  of  the  DVM,  a  practitioner  can  quickly  determine  which  design 
variables  have  the  most  impact  on  general  resource  usage  (in  the  notional  example, 
the  length  and  propeller  type  are  the  most  impactful),  as  well  as  which  resources  are 
more  sensitive  to  the  present  design  choices  (again,  from  the  notional  data,  the 
Acquisition  Cost  is  the  most  sensitive,  followed  by  Lifecycle  Cost  and  Crew  Size). 
Generating  an  enhanced  DVM,  with  both  utility  attributes  and  expense  attributes, 
provides  an  expanded  cost  and  benefit  perspective  on  the  ramifications  of  various 
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design  decisions.  Second,  if  low-impact  design  variables  are  identified  (e.g.,  Draft, 
Deckhouse  Material),  they  can  be  removed  from  the  analysis  to  simplify  the  process 
going  forward  and  concentrate  effort  on  the  design  drivers.  Finally,  these  impacts 
can  be  used  to  inform  the  modeling  and  simulation  of  the  system  necessary  to 
evaluate  system  attributes  in  Process  4,  Design-Epoch  Tradespaces  Evaluation. 

Process  3:  Epoch  Characterization 

After  identification  of  the  design  variables,  performance  and  expense 
attributes,  and  their  corresponding  relationships,  the  internal  and  external 
uncertainties  are  added  into  the  analysis.  Schofield  (2010)  lists  the  external 
uncertainties  (in  the  associated  categories)  related  to  the  OPC  as  follows: 

Technology:  Vertical  Unmanned  Aerial  Vehicle  (VUAV)  integration;  major 
C4ISR  system  upgrade;  and  new  and  more  capable  (size,  range,  personnel 
carried)  small  boats 

Policy:  Marine  engine  emission  reductions;  reduced  copper  content  from 
shipboard  systems  (sea  water  systems);  increased  intelligence  gathering  into 
government-wide  system 

Budget:  Loss  of  acquisition  budget  prior  to  Initial  Operational  Capability 
(IOC);  increase  in  operational  funding  for  increased  operational  usage 

System  of  Systems  (SoS):  Deploying  with  National  Security  Cutters;  new 
cutter-deployed  helicopters 

Missions:  Support  of  arctic  region  for  fisheries;  adding  environmental 
cleanup  response  capability;  more  frequent  international  presence  particularly 
for  peacekeeping  missions 

Epoch  variables  are  generated  from  these  uncertainties  by  determining  the 
primary  source  of  the  possible  changes  in  operating  context.  For  instance,  Schofield 
(2010)  uses  the  marine  engine  emission  reductions  uncertainty  in  the  Policy 
category  to  generate  the  “Engine  Emissions  Rating”  epoch  variable,  which  has  an 
integer  value  range  from  2  to  4.  Due  to  the  similarities  of  operating  contexts  and 
missions,  the  epoch  variables  chosen  for  the  NGCS  are  a  subset  of  those  outlined 
for  the  OPC.  The  epoch  variables  chosen  are  shown  in  Figure  15,  with  the 
corresponding  category,  associated  ranges  of  values,  and  corresponding  units. 
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Levels 

Epoch  Variable 

[Range] 

Units 

EV  -  Tech 

VUAV 

[Small,  Large] 

Storage  Level 

EV  --  Tech 

Small  Boat  Size 

[24,  35] 

ft 

EV  -  Policy 

Engine  Emissions 

[2,  3,  4] 

(Tier) 

EV  --  SoS 

Range  Increase 

[0,  10,  20] 

% 

EV  --Mission 

Ice  Region  Use 

[Low,  Med,  High] 

Level 

Figure  15.  A  List  of  the  Epoch  Variables  Modeled  for  the  NGCS  Context  and 

Needs 

Once  each  epoch  variable  is  created,  the  impact  of  the  epoch  variables  on 
each  of  the  design  variables,  performance  attributes,  and  resource  attributes  can 
then  be  depicted  with  an  Epoch  Descriptor  Impact  Matrix,  similar  to  the  DVM  in 
Process  2.  The  complete  Epoch  Descriptor  Impact  Matrix  with  values  (both  notional 
and  taken  from  Schofield’s  2010  OPC  study)  is  shown  in  Figure  16. 


Epoch  Descriptor  Impact  Matrix 


VUAV 

Small  Boat  Size 

Engine  Emissions 

Range  increase 

Ice  Region  Use 

Total  Impact  j 

Utility  Attributes 

Range 

Speed 

Displacement 

Air  Capability 

Endurance 

Small  Boat  Capability 

1 

1 

1 

9 

9 

21 

1 

1 

9 

9 

3 

23 

1 

1 

O 

3 

3 

3 

3 

3 

0 

3 

3 

12 

1 

1 

1 

9 

3 

15 

1 

9 

1 

3 

3 

17 

Total 

3 

16 

12 

36 

24 

Expense  Attributes 

Acquisition  Cost 
Lifecycle  Cost 

IOC 

Crew  Size 

3 

3 

3 

1 

0 

10 

3 

3 

1 

9 

3 

19 

1 

0 

3 

1 

3 

8 

3 

3 

1 

3 

1 

11 

Total 

10 

9 

3 

14 

7 

Figure  16.  A  Matrix  Reflecting  the  Notional  Impact  of  Epoch  Variables  on 
Design  Variables,  Utility  Attributes,  and  Expense  Attributes 

Similar  conclusions  can  be  drawn  as  in  Process  2;  for  example,  it  is  clear 
from  the  sums  of  rows  in  Design  Variables  that  Speed  and  Range  are  the  utility 
attributes  most  impacted  by  the  uncertainties,  and  Lifecycle  Cost  is  the  most 
impacted  expense  attribute.  Conversely,  the  Range  Increase  epoch  variable  is  the 
most  impactful  (by  quite  a  margin)  on  all  attributes,  with  Ice  Region  Use  heavily 
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impacting  performance.  Gaining  an  understanding  of  these  relationships  early  in  the 
design  process  allows  a  practitioner  to  begin  considering  how  a  design  should  be 
oriented  to  cope  with  uncertainties,  as  well  as  to  keep  in  mind  those  contexts  which 
are  especially  detrimental  to  the  utility  or  expense  of  the  system,  whether  directly  or 
through  opportunity  costs.  In  addition,  this  mapping  will  aid  in  the  evaluation  of 
designs  in  each  epoch  and  era  in  the  subsequent  processes  of  the  method. 

Process  4:  Design-Epoch  Tradespaces  Evaluation 

Once  the  value-driving  context  has  been  defined,  along  with  the  value-driven 
designs  (variables  and  attributes)  and  epochs,  a  practitioner  is  ready  to  begin 
evaluating  candidate  designs  in  all  epochs.  The  evaluation  of  the  potential  designs’ 
attributes  in  the  case  of  the  NGCS  was  achieved  through  use  of  the  MIT  Math 
Model,  which  is  a  set  of  mathematical  relationships  developed  at  MIT  and  used  for 
over  20  years  in  the  design  of  Navy  frigates  for  academic  and  government  studies. 
The  model’s  inputs  and  outputs  include  length,  beam,  draft,  crew  size,  weapon 
packages,  and  many  other  factors  (around  50  in  all).  It  incorporates  detailed 
calculations  of  payload  size,  hull  geometry,  machinery,  power  and  space 
requirements,  weight,  stability,  and  a  simplified  cost  model.  Using  the  model,  a  naval 
subject  matter  expert  generated  six  feasible  ship  designs  based  on  the  design 
variables  provided,  producing  the  attributes  of  Acquisition  and  Lifecycle  Costs,  Crew 
Size,  Range,  Speed,  Displacement,  and  IOC  for  the  six  representative  NGCS 
designs.  These  attributes  were  combined  with  several  others — Air  Capability, 
Endurance,  and  Small  Boat  Capability — along  with  the  notional  impacts  of  the  epoch 
variable  levels  from  Process  3.  The  resulting  (adjusted)  attribute  levels  were  mapped 
to  stakeholder  preferences  through  the  use  of  single  attribute  utility  functions.  The 
utility  curves  for  the  levels  of  each  attribute — normally  captured  through  stakeholder 
interviews,  but  here  generated  through  assessment  of  current  capabilities  and  the 
concepts  of  loss  aversion  and  anchoring  in  prospect  theory  (Kahneman  &  Tversky, 
1979,  1984) — provide  a  single  attribute  utility  value  for  each  system  attribute’s  level. 
The  utility  curves  defined  for  all  attributes  in  the  baseline  epoch  are  shown  in  Figure 
17. 
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Figure  17.  Single  Attribute  Utility  (SAU)  Curves  on  Each  System  Attribute 

Note.  The  leveling  off  of  stakeholder  satisfaction/dissatisfaction  occurs  around  levels 

established  by  previous  systems. 

All  of  the  single  attribute  utility  values  are  then  aggregated  into  a  Multi- 
Attribute  Utility  (MAU)  value  for  each  design  point.  Two  key  assumptions  are  made 
during  this  step:  preferential  independence  among  attributes  and  utility 
independence  among  attributes.  If  each  attribute  of  the  system  contributes 
independently  to  utility,  then  the  swing  weights  (relative  ranking  of  an  attribute’s 
importance  when  it  is  at  its  best  values  and  all  others  are  at  their  minimally 
acceptable  values)  on  each  attribute  sum  to  1 . 

These  assumptions  allow  the  MAU  to  be  calculated  using  a  simple  weighted 
sum  of  the  single  attribute  utilities: 
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(3) 


where 


(4) 


The  MAU  metric  is  commonly  plotted  against  each  design’s  monetary  cost  to 
help  visualize  a  tradespace.  As  the  present  study  is  focused  on  resource  usage, 
however,  monetary  cost  is  replaced  with  the  Multi-Attribute  Expense  (MAE)  metric, 
which  captures  stakeholder  preference  on  other  resource  usage  in  addition  to 
financial  cost  (e.g.,  initial  operating  capability,  crew  size)  through  the  use  of  Single 
Attribute  Expense  (SAE)  functions,  akin  to  the  SAU  and  MAU  functions  described 
previously.  These  preferences  are  shown  in  Figure  18. 


Figure  18.  SAE  Functions  for  the  Expense  Attributes  of  the  NGCS,  Where  a 
Value  of  1  Represents  Complete  Dissatisfaction 

Note.  The  “knees”  in  the  curves  represent  anchoring  from  similar  expenses  of 

previous  systems. 

For  the  present  study,  six  representative  designs  were  chosen  for  evaluation 
throughout  epochs,  shown  in  Figure  19.  The  resulting  evaluations  of  these  designs 
in  the  Baseline  epoch  are  shown  in  Figure  20,  which  lists  all  of  the  attribute  values 
and  the  resulting  MAE  and  MAU  values  for  each  design. 
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Blue  Diam. 

Red  Square 

Green  Triangle 

Purple  X 

Blue  X 

Yellow  Circle 

Design  #: 

1 

2 

3 

4 

5 

6 

Length  (ft) 

519 

530 

440 

521 

510 

562 

Beam  (ft) 

59.8 

59 

51.1 

57.5 

56.8 

62.2 

Draft  (cu.ft.) 

19.1 

18.8 

16.3 

18.3 

20.3 

19.4 

Prop  Type 

CRPP 

CRPP 

CRPP 

CRPP 

CRPP 

CRPP 

Hull  Material 

OS 

OS 

HTS 

HTS 

OS 

OS 

DH  Material 

Aluminum 

Steel 

Aluminum 

Aluminum 

Steel 

Aluminum 

Def  Capability 

Low 

High 

Low 

Low 

High 

Low 

Helos 

2 

1 

1 

2 

2 

2 

ASUW 

High 

Medium 

Low 

Medium 

Medium 

High 

AAW 

High 

Medium 

Low 

Medium 

Medium 

High 

Figure  19.  The  Six  Representative  Designs  (With  Corresponding  Design 
Variable  Levels)  for  the  Initial  Concept  Selection  of  the  NGCS 


EPOCH:  Baseline 

Blue  Diam.  Red  Square  Green  Triangle  Purple  X  Blue  X  Yellow  Circle 

Design#: 

1 

2 

3 

4 

5 

6 

Acquisiton 

1156 

968 

627 

1011 

1009 

1230 

Lifecycle 

5086 

4364 

3244 

4510 

4505 

5263 

IOC 

2018 

2018 

2018 

2018 

2018 

2018 

Crew  Size 

225 

230 

200 

225 

220 

250 

Displacement 

7663 

8143 

4654 

7734 

8639 

9800 

Range 

4500 

6500 

4500 

6500 

8000 

6500 

Speed 

20 

20 

30 

30 

30 

40 

AIR  CAP 

8 

4 

2 

8 

8 

12 

ENDRNC 

50 

50 

30 

45 

45 

60 

SB  CAP 

60 

20 

20 

60 

40 

100 

MAE 

0.36 

0.28 

0.13 

0.29 

0.23 

0.41 

MAU 

0.59 

0.51 

0.31 

0.73 

0.73 

0.90 

Figure  20.  The  Evaluated  Attributes  of  the  Six  Representative  Designs  in  the 
Baseline  Epoch,  With  Aggregated  MAE  and  MAU  Values 

Note.  See  Figure  2  for  units  of  measurement. 


One  of  the  visual  results  of  the  MAU  and  MAE  evaluations  in  the  Baseline 
Epoch  is  a  tradespace  plotting  the  MAU  versus  MAE  metrics  for  the  handful  of 
NGCS  designs,  shown  in  Figure  21.  These  evaluations  are  performed  for  all  designs 
in  all  epochs  (a  representative  six  epochs,  in  the  case  of  the  NGCS),  providing  the 
metrics  necessary  for  the  remaining  steps  of  the  method.  Different  sets  of 
stakeholder  preferences  on  utility  attributes  were  used  in  addition  to  those  shown  in 
Figure  17,  depending  on  epoch. 
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Baseline  (N=6) 


MAE 


Figure  21 .  One  of  the  Results  of  Attribute  Evaluation  Through  the  Math 
Model,  Epoch  Variable  Impacts,  and  Single/Multi-Attribute  Utilities:  A 
Tradespace  (MAU  vs.  MAE)  of  Six  NGCS  Designs  Operating  in  the 

Baseline  Epoch 

The  six  epochs  evaluated  for  the  NGCS  study  are  shown  in  Figure  22.  They 
represent  all  of  the  108  possible  epochs  from  the  combinations  of  the  epoch  variable 
levels  (see  Figure  16). 


VUAV 

Small  Boat  Size 

Engine  Emissions 

Range  Increase 

Ice  Region  Use 

Conflict 

Large 

35 

4 

20 

High 

Mothership 

Large 

35 

2 

10 

Low 

Sojourning 

Small 

24 

2 

20 

High 

Sea  Support 

Small 

35 

3 

10 

Medium 

Non-Polluter 

Small 

24 

4 

0 

Low 

Baseline 

Small 

24 

2 

0 

Low 

Figure  22.  The  Six  Representative  Epochs  Constructed  for  the  NGCS  Study 

(of  108  Possible) 


Process  5:  Single-Epoch  Analyses 

Once  the  evaluations  of  Process  4  are  complete  for  all  designs  in  all  epochs, 
analysis  of  design  characteristics  in  single  epochs  can  be  performed.  In  addition  to 
the  MAU  and  MAE  metrics,  the  present  case  considers  the  monetary  cost  of  each 
design  as  well  as  each  design’s  Pareto  efficiency  in  the  tradespace.  This  analysis 
can  be  repeated  for  any  number  of  epochs  of  interest,  which  can  be  chosen  through 
various  means — those  most  likely  to  occur,  those  most  likely  to  hinder  value 
delivery,  or  those  of  concern  for  other  reasons  to  stakeholders  and  analysts.  The 
present  study  chooses  only  a  few  of  the  epochs  created  in  Process  4:  Mothership, 
Sea  Support,  and  Sojourner. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-30- 


Mothership  is  an  epoch  characterized  by  Large  VUAVs  and  35ft  Small  Boats, 
with  a  10%  increase  in  Range  of  mission  over  the  Baseline  Epoch.  This  combination 
of  epoch  variables  represents  a  period  in  which  stakeholders  would  desire  the 
NGCS  to  support  air  and  sea  operations  over  non-Arctic  waters.  The  evaluated 
designs  for  this  epoch  are  shown  in  Figure  23,  and  the  corresponding  tradespace  is 
depicted  in  Figure  24. 


Mothership 

Blue  Dfam.  Red  Square  Green  Triangle  Purple  X  Blue  X  Yellow  Circle 

Design  #: 

1 

2 

3 

4 

5 

6 

Acquisiton 

1274 

1067 

691 

1115 

1112 

1356 

Lifecycle 

6167 

5293 

3934 

5470 

5464 

6382 

IOC 

2018 

2018 

2018 

2018 

2018 

2018 

Crew  Size 

250 

255 

225 

250 

245 

275 

Displacement 

7663 

8143 

4654 

7734 

8639 

9800 

Range 

4500 

6500 

4500 

6500 

8000 

6500 

Speed 

18 

IS 

27 

27 

27 

36 

AIR  CAP 

0 

-4 

-6 

0 

0 

4 

ENDRNC 

45 

45 

25 

40 

40 

55 

SB  CAP 

50 

10 

10 

50 

30 

90 

MAE 

0.47 

0.36 

0.21 

0.39 

0.38 

0.51 

MAU 

0.38 

0.18 

infeasible 

0.54 

0.53 

0.75 

Figure  23.  The  Evaluated  Attributes  of  the  Six  Representative  NGCS  Designs 

in  the  Mothership  Epoch 

Note.  See  Figure  2  for  units  of  measurement;  negative  attribute  values  are  treated  as  0. 


Mothership  (N=5) 


MAE 


Figure  24.  The  Six  Potential  NGCS  Designs  in  the  Mothership  Epoch 

Note .  One  design  is  below  minimum  acceptable  utility,  leaving  the  five  feasible  designs. 

It  is  important  to  note  that  in  general,  the  MAU  and  MAE  values  cannot  be 
compared  between  epochs,  since  stakeholder  preferences  may  change  between 
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epochs  (i.e.,  changed  what  the  0-to-1  scale  represents,  as  it  is  not  a  universal 
scale).  The  yields  of  the  tradespaces  can  always  be  compared,  however,  providing 
the  number  of  designs  at  or  above  the  minimum  acceptable  utility  and  at  or  below 
maximum  acceptable  expense.  While  six  designs  were  evaluated  in  each  epoch, 
only  five  are  feasible  in  the  Mothership  epoch,  indicating  that  this  epoch  provides 
some  challenges  for  at  least  one  potential  design  (Design  #3)  to  provide  minimum 
acceptable  utility  and/or  operate  below  maximum  acceptable  expense. 

Because  acquisition  and  lifecycle  costs  can  be  limiting  factors  compared  to 
the  other  expenses  (e.g.,  crew  size)  rolled  up  in  the  MAE  metric,  the  monetary  costs 
of  each  design  are  briefly  observed  in  each  epoch  of  interest.  Design  #6  has  an 
acquisition  cost  of  $1 .3  billion  and  a  lifecycle  cost  of  $6.4  billion,  while  Design  #2 
design’s  acquisition  cost  is  30%  less  (~$1  billion),  and  its  lifecycle  cost  is  around 
15%  less  ($5.3  billion).  The  other  designs’  costs  are  in  the  middle  of  these  two 
designs.  If  budget  levels  were  established  for  the  stakeholder  in  this  epoch,  those 
considerations  could  aid  in  the  comparison  of  these  costs.  In  addition,  Process  6  will 
examine  more  informative  cost  metrics  over  all  epochs,  with  or  without  established 
budget  levels. 

The  Fuzzy  Pareto  Number  (FPN)  is  a  metric  developed  to  indicate  a  design’s 
relative  value  in  a  given  epoch,  as  it  measures  how  far  from  Pareto-optimality  that 
design  lies  in  the  tradespace  of  that  epoch  (Fitzgerald  &  Ross,  2012).  Since  it 
represents  a  percentage  of  deviation  from  cost/utility  Pareto  efficiency,  it  is 
measured  from  0  to  100;  in  addition,  since  it  is  a  percentage,  it  can  be  compared 
across  epochs  as  an  indicator  of  relative  efficiency  differences.  In  the  Mothership 
epoch,  the  FPN  for  four  of  the  designs  is  0  due  to  their  locations  on  the  Pareto  front. 
The  remaining  design,  Design  #1 ,  has  an  FPN  number  of  20,  meaning  that  it  is  20% 
“inefficient”  compared  to  the  Pareto  front  in  this  epoch. 

These  analyses  are  now  briefly  discussed  for  the  rest  of  the  epochs  of 
interest. 

Sea  Support  is  an  epoch  during  which  the  NGCS  would  be  required  to 
support  extended  missions  with  very  capable  small  boats  over  a  wide  range  of 
global  waters.  It  assigns  the  following  values  to  the  epoch  variables  (various 
possible  levels  described  in  Process  3):  Small  Boat  Size  of  35ft,  Emissions 
standards  at  Level  3,  Range  Increase  of  10%,  and  High  Ice  Region  Use.  The 
evaluated  attributes  of  all  designs  are  shown  in  Figure  25;  the  tradespace  of  MAU 
versus  MAE  follows  in  Figure  26. 
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Sea  Support 

Blue  Diam.  Red  Square  Green  Triangle  Purple  X  Blue  X  Yellow  Circle 

Design  #: 

1 

2 

3 

4 

5 

6 

Acquisiton 

1274 

1067 

691 

1115 

1112 

1356 

Lifecycle 

5887 

5052 

3755 

5221 

5216 

6092 

IOC 

2020 

2020 

2020 

2020 

2020 

2020 

Crew  Size 

240 

245 

215 

240 

235 

265 

Displacement 

7663 

8143 

4654 

7734 

8639 

9800 

Range 

3600 

5200 

3600 

5200 

6400 

5200 

Speed 

15 

15 

23 

23 

23 

31 

AIR  CAP 

2 

-2 

-4 

2 

2 

6 

ENDRNC 

40 

40 

20 

35 

35 

50 

SB  CAP 

40 

0 

0 

40 

20 

80 

MAE 

0.70 

0.60 

0.44 

0.62 

0.61 

0.75 

MAU 

0.14 

0.15 

infeasible 

0.46 

0.46 

0.71 

Figure  25.  The  Evaluated  Attributes  of  Each  Design  in  the  Sea  Support 

Epoch 

Note.  See  Figure  2  for  units  of  measurement;  negative  attribute  values  are  treated  as  0. 


Sea  Support  (N=5) 

l.UU 

0.80 

• 

0.60 

D 

< 

% 

0.40 

0.20 

B  # 

0.00 

0.1 

1  1-^  1  1  1 

DO  0.20  0.40  0.60  0.80  1.00 

MAE 

Figure  26.  The  Four  Feasible  NGCS  Designs  in  the  Epoch  Sea  Support 

The  yield  of  this  epoch  is,  again,  five  out  of  the  six  evaluated  designs.  From 
Figure  25,  we  can  see  that  the  acquisition  costs  are  similar  to  that  of  the  Mothership 
epoch,  but  the  lifecycle  cost  is  slightly  less  in  this  epoch.  We  can  also  see  that  the 
IOC  is  pushed  back  to  the  year  2020  for  all  designs.  The  FPNs  for  Designs  #2,  #4, 
#5,  and  #6  are,  once  again,  0,  while  the  FPN  for  Design  #1  is  38. 

The  third  epoch  of  interest  considered,  Sojourner,  is  defined  by  a  Range 
Increase  of  20%  plus  High  Ice  Region  Use.  The  stakeholder’s  preferences  in  this 
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epoch  are  agnostic  of  the  air  and  small  boat  capabilities  of  each  design.  Figure  27 
displays  the  evaluated  attributes  of  each  design  in  this  epoch,  and  the  tradespace  of 
MAU  versus  MAE  is  shown  in  Figure  28. 


Sojourner 

Blue  Diam.  Red  Square  Green  Triangle  Purple  X  Blue  X  Yellow  Circle 

Design  #: 

1 

2 

3 

4 

5 

6 

Acquisiton 

1156 

968 

627 

1011 

1009 

1230 

Lifecycle 

6141 

5270 

3917 

5446 

5440 

6355 

IOC 

2019 

2019 

2019 

2019 

2019 

2019 

Crew  Size 

245 

250 

220 

245 

240 

270 

Displacement 

7663 

8143 

4654 

7734 

8639 

9800 

Range 

3600 
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ENDRNC 
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25 

40 

SB  CAP 

35 

-5 

-5 

35 

15 

75 

MAE 

0.51 

0.41 

0.27 

0.43 

0.42 

0.55 

MAU 

0.04 

0.33 

infeasible 

0,23 

0.27 

0.59 

Figure  27.  The  Evaluated  Attributes  of  Each  Design  in  the  Sojourner  Epoch 

Note.  See  Figure  2  for  units  of  measurement;  negative  attribute  values  are  treated  as 

zero. 


Sojourner  (N=5) 


MAE 


Figure  28.  The  Five  Feasible  Potential  NGCS  Designs  in  the  Sojourner  Epoch 

The  yield  of  this  epoch  is  five  of  the  six  designs;  only  Design  #3  is  infeasible. 
From  Figure  27  we  can  see  that  the  acquisition  costs  are  similar  to  that  of  the 
previous  epochs  considered,  but  the  lifecycle  costs  of  all  designs  are  around  the 
Mothership  epoch’s  levels.  It  is  of  interest  to  note  that  only  two  designs  remain  on 
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the  Pareto  front:  Designs  #2  and  #6.  The  FPN  for  Design  #4  is  4,  the  FPN  for  Design 
#5  is  3,  and  Design  #1’s  FPN  is  23. 

Figure  29  shows  each  of  the  six  epoch  tradespaces  of  the  potential  NGCS 
designs  side  by  side  for  easier  comparison. 
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Figure  29.  All  Six  Epoch  Tradespaces  for  NGCS 
Process  6:  Multi-Epoch  Analysis 

The  analyses  of  Process  5  can  be  enlightening  regarding  the  behavior  of 
individual  designs  and  the  impacts  of  individual  epochs,  but  the  observations  can  be 
time  consuming  (and  the  resulting  data  overwhelming)  for  any  significant  number  of 
designs  and/or  epochs.  The  analyses  can  also  allow  undue  weight  to  be  placed  on 
the  epochs  of  interest  over  those  omitted  from  explicit  consideration,  even  though  all 
epochs  represent  possible  operating  environments.  For  this  reason,  Process  6 
focuses  on  several  summary  metrics  to  gain  a  higher-level  view  of  all  designs’ 
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characteristics  over  all  epochs.  Since  only  six  epochs  are  used  as  representative  of 
all  epochs  in  this  study,  these  tradespaces  are  shown  side  by  side  for  easy 
comparison  in  Figure  29.  No  weight  is  given  to  epochs  based  on  likelihood  of 
occurrence,  as  the  purpose  of  this  analysis  is  simply  to  cover  all  possible  scenarios 
the  system  might  encounter.  The  first  of  the  metrics  designed  for  this  purpose  is  the 
Normalized  Pareto  Trace. 

The  Normalized  Pareto  Trace  (NPT)  reflects  the  percentage  of  all  epochs  for 
which  a  given  design  is  Pareto  efficient  (Ross,  Rhodes,  &  Hastings,  2009).  A  higher 
NPT  indicates  higher  Pareto  efficiency  for  a  design  over  all  epochs,  rather  than 
higher  Pareto  efficiency  in  only  one  epoch  (like  the  FPN).  It  is  calculated  for  any 
design  by  counting  the  number  of  epochs  in  which  that  design  has  an  FPN  of  zero 
and  then  dividing  by  the  total  number  of  epochs.  The  six  potential  NGCS  designs 
and  their  corresponding  NPTs  across  the  six  representative  epochs  (shown  in  Figure 
22)  are  listed  in  Figure  30. 


Design 

# 

NPT 

Blue  Diamond 

1 

0 

Red  Square 

2 

1 

Green  Triangle 

3 

0.33 

Purple  X 

4 

0.5 

Blue  X 

5 

0.67 

Yellow  Circle 

6 

1 

Figure  30.  The  NPTs  for  the  Six  NGCS  Designs  Across  the  Six 

Representative  Epochs 

Note.  An  NPT  value  of  1  represents  Pareto  efficiency  in  100%  of  the  epochs. 

Clearly,  Designs  #2  and  #6  would  be  good  choices  when  high  priority  is 
placed  on  Pareto  efficiency,  as  they  remain  on  the  Pareto  front  no  matter  which 
epoch  is  encountered.  Design  #6  is  the  most  expensive  and  brings  the  most  utility  in 
every  epoch,  while  Design  #2  is  one  of  the  lower  expense  designs  and  is  closer  to 
minimal  acceptable  utility  in  most  epochs. 

The  somewhat  simplistic  measure  of  NPT  can  be  extended  by  allowing  some 
“fuzziness”  threshold  in  the  evaluation  of  efficiency.  The  fuzzy  Normalized  Pareto 
Trace  (fNPT)  is  a  metric  that  does  precisely  this — applies  a  specified  fuzziness 
percentage  to  the  Pareto  front  in  each  epoch,  where  0%  is  a  normal  Pareto  front, 
and  100%  includes  the  entire  range  of  both  the  MAU  and  MAE  of  the  designs  in  a 
given  epoch  (Fitzgerald,  Ross,  &  Rhodes,  2012).  The  fNPT  of  each  design  at 
several  fuzziness  levels  is  shown  in  Figure  31 . 
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Design 

# 

0%  fNPT 

5%  fNPT 

10%  fNPT 

20%  fNPT 

Blue  Diamond 
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0.17 

Red  Square 
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1 

Green  Triangle 
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0.33 

0.33 

0.33 

0.33 

Purple  X 

4 

0.5 

1 

1 

1 

Blue  X 

5 

0.67 

1 

1 

1 

Yellow  Circle 

6 

1 

1 

1 

1 

Figure  31 .  The  Fuzzy  NPTs  of  the  Six  NGCS  Designs  for  Several  Levels  of 

Fuzziness 

Note.  A  value  of  1  represents  fuzzy  Pareto  efficiency  in  1 00%  of  the  epochs. 


The  results  of  the  fNPTs  show  that  with  a  small  amount  of  fuzziness — around 
5% — most  of  the  NGCS  designs  have  maxed  out  their  fNPT  number  (due  in  part  to 
the  small  number  of  designs  considered).  Design  #1  remains  inefficient  even  after 
the  fuzziness  level  approaches  20%,  decreasing  this  design’s  attractiveness 
compared  to  the  others.  With  more  designs  under  consideration,  the  benefits  of  the 
fNPT  metric  should  be  evident:  helping  identify  designs  that  may  be  near-Pareto 
efficient  in  many  epochs  and  resultantly  missed  by  the  original  NPT  (such  as 
Designs  #4  and  #5  in  this  case). 

Aspects  of  changeability 

The  metrics  shown  so  far  do  not  take  into  consideration  the  possibility  of 
changing  from  one  design  to  another  in  a  given  epoch.  This  possibility  is  usually 
present  in  some  form,  however,  and  as  a  result  it  is  helpful  for  multi-epoch  analyses 
to  evaluate  a  design  based  on  the  strategies  used  for  change — that  is,  an  original 
design  can  be  evaluated  in  multi-epoch  analyses  by  evaluating  the  target  design  (to 
which  the  original  changes,  if  applicable)  in  each  epoch.  Strategies  can  be  defined  to 
guide  the  change  behavior;  for  the  NGCS  case,  the  strategy  chosen  was  “Maximize 
Efficiency”  (i.e.,  move  to  the  Pareto  front  if  not  already  there).  Transition  rules  were 
created  such  that  any  design  could  be  discarded  and  any  design  purchased  in  any 
epoch.  A  transition  matrix  was  then  constructed  to  reflect  the  resulting  target  design 
(dictated  by  the  chosen  strategy)  in  each  epoch  from  each  original  design  in  that 
epoch.  With  this  additional  information  on  the  anticipated  transitions  and  resulting 
designs  in  each  epoch,  modified  forms  of  the  previous  metrics  were  constructed  and 
are  now  discussed. 

The  effective  NPT  (eNPT)  and  effective  fNPT  (efNPT)  metrics  evaluate  a 
design  across  all  epochs  in  the  following  way:  If  the  change  strategy  dictates  that  a 
(original)  design  changes  to  another  (target)  design  in  a  given  epoch,  then  the  target 
design  is  evaluated;  if  the  change  strategy  dictates  that  a  starting  design  does  not 
change  in  that  epoch,  then  that  starting  design  is  evaluated  (Fitzgerald  &  Ross, 
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2012).  The  eNPTs  for  the  NGCS  designs,  generated  by  the  change  strategy 
“Maximize  Efficiency”  discussed  previously,  are  shown  in  Figure  32. 
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0.67 
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1 

Figure  32.  The  eNPTs  for  the  NGCS  Designs  With  the  Change  Strategy 

“Maximize  Efficiency” 


As  the  table  shows,  most  designs’  base  eNPT  is  a  great  improvement  over 
the  NPT.  In  fact,  it  appears  that  every  design  can  achieve  Pareto  efficiency  in  every 
epoch.  This  is  true  because  of  the  transition  rules  defined  and  the  transition  strategy 
chosen.  Recall  that  the  transition  rules  defined  for  the  NGCS  case  state  that  every 
design  can  be  discarded  and  any  other  design  purchased  in  its  place  in  any  epoch, 
while  the  transition  strategy  dictates  movement  to  the  Pareto  front  in  every  epoch. 

If  smaller  scale  changes  than  discard/replace  were  available  (e.g.,  “add  UAV 
storage,”  “remove  crew,”  etc.),  rules  could  be  established  to  represent  the 
feasibility/infeasibility  of  changing  from  one  design  to  another  in  any  epoch,  thereby 
limiting  the  improvement  of  a  design’s  eNPT  over  its  NPT.  Likewise,  transition  costs 
of  money  and  time  could  be  defined  for  any  of  the  transitions  between  designs,  and 
these  costs  could  be  used  to  limit  feasibility  (depending  on  the  budget/goal  of  the 
transition  strategy  defined)  as  well  as  to  track  the  total  expenditures  necessary  to 
achieve  a  given  eNPT.  In  the  notional  columns  on  the  right  of  Figure  32,  it  can  be 
seen  that  budgets  of  time  or  financial  costs  would  prevent  Design  #1 ,  for  instance, 
from  improving  very  much.  Likewise,  the  other  designs  may  improve  or  not.  (The 
effective  fNPT  [efNPT]  metric  can  also  be  constructed,  but  is  left  out  of  the  present 
study  due  to  the  already-maximized  eNPT  values  above.) 

Additional  aspects  of  affordability 

In  addition  to  the  efficiency  of  a  design  relative  to  other  designs  in  each 
epoch,  it  can  be  useful  to  consider  the  resource  expenditures  of  a  design  in  various 
operating  environments.  Two  ways  of  measuring  expenditures  across  all  epochs  are 
applied  to  the  NGCS  case:  One  tracks  the  maximum  amount  required  of  each 
resource  for  a  given  design,  and  the  other  tracks  the  stability  of  resource 
consumption  throughout  all  epochs.  The  first  metric  can  help  identify  designs  that 
would  be  unsustainable  given  the  right  conditions,  while  the  second  metric  can 
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identify  designs  for  which  allocating  resources  in  the  future  (e.g.,  congressional 
budget  requests)  may  prove  easier  due  to  consistency  through  changing  operational 
environments.  In  the  case  of  the  NGCS,  the  highest  levels  of  expense  for  each 
design’s  Lifecycle  Cost  and  Crew  Size  in  all  (six  representative)  epochs  are  shown 
in  Figure  33. 


Max  Expense  Design 
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Lifecycle  Cost  [$  mil.) 
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6,087 
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260 
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285 

Figure  33.  The  Highest  Levels  of  Lifecycle  Cost  and  Crew  Size  Across  All 

Epochs 

Recalling  the  Normalized  Pareto  Trace  from  earlier,  efficient  Designs  #2  and 
#6  can  be  selected  for  comparison  of  their  respective  maximum  expenses  incurred. 
While  Design  #2  could  cost  a  maximum  of  just  over  $6  billion  in  its  lifetime,  Design 
#6  could  cost  over  20%  more  in  one  possible  scenario.  In  addition,  Design  #6  could 
require  25  more  crewmen  depending  on  the  epoch  encountered.  Note  that  multiple 
designs’  maximum  expenses  are  not  all  necessarily  from  the  same  epoch,  as  epoch 
variables  impact  individual  designs  differently.  (Likewise,  with  a  design’s  multiple 
expense  attributes,  each  of  a  design’s  attributes  are  impacted  differently  by  the 
epoch  variables,  so  that  the  lifecycle  cost  may  be  highest  in  one  epoch,  while 
maximum  crew  size  is  required  in  another.) 

The  standard  deviation  of  each  design’s  lifecycle  cost  is  shown  in  Figure  34. 
Continuing  our  analysis  from  the  Max  Expense,  it  can  be  seen  that  Design  #2  has 
somewhat  less  variability  across  epochs  than  Design  #6,  by  around  $400  million.  If 
budget  forecasts  were  known  at  the  time  of  this  analysis,  such  information  could  be 
used  effectively  to  choose  a  design  that  fit  within  the  expected  variability  of  the 
budget  for  the  timeframe  of  the  forecast. 


Expense  Stability  Design 

(St.Dev.  in  $  mil.) 
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2 
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4 

5 

6 

Lifecycle  Cost 

2,156 

1,850 

1,375 

1,912 

1,910 

2,231 

Figure  34.  The  Standard  Deviation  of  Lifecycle  Cost  for  Each  Design  Across 

All  Epochs 

These  considerations,  when  combined  with  previous  analyses,  help  outline 
the  impact  of  the  risks  involved  for  any  initial  concept  selection,  and  they  can  help 
analysts  and  decision-makers  alike  understand  the  traits  behind  design  concepts 
that  would  be  affordable  in  all  futures. 
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Process  7:  Era  Construction 

The  analysis  up  to  this  point  has  evaluated  NGCS  designs  in  epochs,  or 
short-run  periods  of  fixed  contexts  and  needs.  By  combining  these  short-run  fixed 
periods  into  a  sequence  over  a  longer  period  of  time,  an  era  is  created — allowing  a 
practitioner  to  study  the  attributes  of  designs  over  one  possible  development  of  the 
operating  environment,  as  well  as  the  effects  of  path  dependence  through  epochs 
(Ross  &  Rhodes,  2008).  Two  eras  were  manually  constructed  for  the  present  study, 
each  representing  a  10-year  sequence  of  epochs  that  the  NGCS  may  encounter 
(taken  from  the  six  representative  epochs  from  Process  4,  c.f.  Figure  22).  The  first 
era  considered  consists  of  the  following  sequence: 

Baseline  (36  months),  Sea  Support  (36  months),  Baseline  (24  months),  Non- 
Polluting  (24  months). 

The  second  era  considered  consists  of 

Sojourning  (24  months),  Conflict  (36  months),  Mothership  (36  months),  and 
Sojourning  (24  months). 

Process  8:  Single-Era  Analyses 

The  first  era  comprises  epochs  in  which  stakeholder  preferences  do  not 
change.  As  a  result,  the  MAE  and  MAU  values  can  be  compared  directly.  The  era  is 
shown  in  Figure  35  (each  tradespace)  and  Figure  36  (line  graph  view). 


(36  months)  (36  months)  (24  months) 


(24  months) 


Figure  35.  The  Four  Epochs  of  Era  #1  and  Their  Durations 

Note.  Stakeholder  preferences  remain  constant;  as  a  result,  MAU  and  MAE  values  can 

be  directly  compared  across  epochs. 
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Figure  36.  Left:  The  MAU  Values  of  Each  Design  During  Era  #1;  Right:  The 

MAE  Values  of  Each  Design  in  Era  #1 


Before  proceeding,  it  should  be  noted  that  Design  #3  is  infeasible  in  the  Sea 
Support  epoch,  meaning  that  design  cannot  provide  minimum  acceptable  utility 
throughout  this  era.  If  a  decision-maker  believes  this  era  to  be  one  of  the  more  likely 
narratives  to  play  out  for  the  NGCS,  then  that  design  should  be  removed  from 
consideration. 

Using  the  concept  of  time  value  of  money,  the  NPV  of  each  design’s 
operations  cost  can  be  calculated  for  the  entire  era.  (To  obtain  the  operations  cost 
for  a  given  timeframe,  the  acquisition  cost  was  subtracted  from  the  30-year  lifecycle 
cost,  and  the  result  divided  into  the  appropriate  number  of  months.)  For  instance, 
Design  #1’s  yearly  operations  cost  in  the  Baseline  Epoch  is  $131  million;  this 
amount  is  used  as  the  input  to  the  final  NPV  calculation.  This  same  calculation  is 
performed  for  each  design  in  each  epoch  in  the  era,  and  a  10%  discount  rate  is 
assumed.  The  intermediate  values  and  final  calculation  are  shown  in  Figure  37. 


Era  #1 


($  millions) 

Design  1 

Design  2 

Design  3 

Design  4 

Design  5 

Design  6 

NPV  Ops  Yr.  1 

131 

113 

87 

117 

117 

134 

NPV  Ops  Yr.  2 

119 

103 

79 

106 

106 

122 

NPV  Ops  Yr.  3 

108 

94 

72 

96 

96 

111 

NPV  Ops  Yr.  4 

116 

100 

77 

103 

103 

119 

NPV  Ops  Yr.  5 

105 

91 

70 

93 

93 

108 

NPV  Ops  Yr.  6 

95 

82 

63 

85 

85 

98 

NPV  Ops  Yr.  7 

74 

64 

49 

66 

66 

76 

NPV  Ops  Yr.  8 

67 

58 

45 

60 

60 

69 

NPV  Ops  Yr.  9 

59 

51 

40 

53 

53 

61 

NPV  Ops  Yr.  10 

54 

47 

36 

48 

48 

55 

NPV  Total: 

929 

803 

618 

827 

826 

953 

Figure  37.  Calculation  of  the  Net  Present  Value  ($  Millions)  of  Designs' 

Operations  Costs  for  Era  #1 

From  these  numbers,  it  appears  that  Designs  #2,  #4,  and  #5  form  one  group 
with  similar  NPV  operations  costs,  and  that  Designs  #1  and  #6  form  another  similar 
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group.  While  these  numbers  reflect  the  same  grouping  of  designs  as  the  MAE 
suggests  in  the  tradespace  plots,  it  is  important  to  note  that  the  MAEs  could  be 
similar  for  certain  designs  whose  operations  cost’s  NPV  are  vastly  different.  For  this 
reason,  it  can  be  necessary  to  consider  these  costs  separately  from  the  MAE  score 
itself  if  information  about  the  operations  budget  is  known  at  this  stage. 

The  additional  affordability  considerations  from  Process  6  can  also  be 
repeated  at  this  point:  The  maximum  values  of  the  operations  costs  of  each  design 
are  shown  in  Figure  38,  and  the  standard  deviation  of  operations  costs  throughout 
the  era  is  shown  in  Figure  39. 


($  million  /  yr) 

Design  1 

Design  2 

Design  3 

Design  4 

Design  5 

Design  6 

Max  Ops  Cost: 

154 
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158 

Max  Ops  Cost  (NPV): 

131 

113 

87 

117 

117 

134 

Figure  38.  The  Maximum  Yearly  Operations  Cost  of  Each  Design  Throughout 
Era  #1,  Compared  With  the  NPV  Maximum  Operations  Cost 


(St.Dev.  in  $  mil.) 

Design  1 

Design  2 

Design  3 

Design  4 

Design  5 

Design  6 

Expense  Variability: 

11.0 

9.5 

7.1 

9.8 

9.8 

11.4 

Figure  39.  The  Standard  Deviation  of  Each  Design’s  Operations  Costs 

Throughout  Era  #1 

In  this  case,  the  results  from  the  era  analysis  reflect  the  same  conclusion  as 
Process  6  (over  all  epochs):  The  more  expensive  designs  have  slightly  more 
variance  than  the  less  expensive  designs.  If  more  designs  were  under  consideration, 
it  may  be  possible  to  find  a  more  expensive  design  that  is  more  cost-stable  across 
all  epochs  (and/or  in  a  particular  era),  which  may  give  incentive  to  include  it  when 
considering  designs  for  final  selection.  Of  course,  these  metrics  can  be  considered 
for  any  resources  of  particular  concern  but  are  only  applied  here  to  operations  cost 
for  demonstration  purposes. 

The  second  era,  shown  in  Figure  40  consists  of  epochs  with  changing 
stakeholder  preferences,  and  so  the  MAU  and  MAE  values  cannot  be  compared 
directly  across  epochs. 
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(24  months) 


(36  months) 


(36  months) 


(24  months) 


Figure  40.  The  Four  Epochs  of  Era  #2 

Note.  The  MAU  and  MAE  values  cannot  be  compared  between  epochs  due  to  changing 

stakeholder  preferences. 


Figure  41.  Left:  The  MAU  of  Each  Design  Through  Era  #2;  Right:  The  MAE  of 

Each  Design  During  the  Same  10-Year  Period 

Note.  Stakeholder  preferences  change  between  epochs. 

It  is  again  noted  that  Design  #3  is  infeasible  in  this  era;  in  fact,  it  is  not 
feasible  in  even  one  of  these  epochs.  Design  #2,  while  starting  the  era  highly  ranked 
in  the  MAU  measurement,  makes  its  way  to  the  least  favorable  ranking  for  a  short 
time.  Its  expenses  remain  least  dissatisfying,  however,  among  feasible  designs  in 
this  era.  The  NPV  calculations  for  all  designs  in  this  era  are  shown  in  Figure  42. 
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Era  #2 


($  millions) 

Design  1 

Design  2 

Design  3 

Design  4 

Design  5 

Design  6 

NPV  Ops  Yr.  1 

166 

143 

110 

148 

148 

171 

NPV  Ops  Yr.  2 

151 

130 

100 

134 

134 

155 

NPV  Ops  Yr.  3 

157 

135 

104 

140 

139 

161 

NPV  Ops  Yr.  4 

143 

123 

94 

127 

127 

146 

NPV  Ops  Yr.  5 

130 

112 

86 

115 

115 

133 

NPV  Ops  Yr.  6 

101 

87 

67 

90 

90 

104 

NPV  Ops  Yr.  7 

92 

80 

61 

82 

82 

95 

NPV  Ops  Yr.  8 

84 

72 

55 

74 

74 

86 

NPV  Ops  Yr.  9 

78 

67 

51 

69 

69 

80 

NPV  Ops  Yr.  10 

70 

61 

47 

63 

63 

72 

NPV  Total: 

1171 

1011 

774 

1042 

1041 

1204 

Figure  42.  Calculation  of  the  NPV  Operations  Costs  of  Each  Design  in  the 

10-Year  Era  #2 

It  should  be  obvious  that  this  era  represents  a  particularly  challenging 
environment  for  the  designs,  with  two  of  them  exceeding  one  billion  dollars  for  the 
NPV  operation  costs.  The  maximum  values  and  the  standard  deviations  of 
operations  costs  for  all  designs  are  shown  in  Figure  43  and  Figure  44. 


($  million  /  yr) 

Design  1 

Design  2 

Design  3 

Design  4 

Design  5 

Design  6 

Max  Ops  Cost: 

190 

164 

125 

169 

169 

195 

Max  Ops  Cost  (NPV): 

166 

143 

110 

148 

148 

171 

Figure  43.  The  Maximum  Operations  Cost  Incurred  by  Each  Design 
Throughout  Era  #2,  Compared  With  the  Maximum  NPV  Operations 

Cost 


(St.Dev.  in  $  mil.) 

Design  1 

Design  2  Design  3  Design  4 

Design  5  Design  6 

Expense  Variability: 

11.5 

9.9  7.6  10.2 

10.2  11.8 

Figure  44.  The  Standard  Deviation  of  Operations  Cost  Throughout  the  10 

Years  of  Era  #2 


Process  9:  Multi-Era  Analysis 

While  certain  insights  can  be  gained  by  inspection  of  individual  eras  such  as 
those  in  Process  8,  it  is  also  instructive  to  observe  alternatives’  behavior  across  the 
era  space  (i.e.,  in  many  or  all  of  the  eras  constructed  by  Process  7)  since  large 
numbers  of  eras  can  be  constructed  through  combinatorics,  probabilistic  state 
transitions,  and  other  means.  Process  9  focuses  on  the  analysis  across  many  eras, 
which  are  somewhat  analogous  to  the  Multi-Epoch  Analysis  of  Process  6. 
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Most  of  the  metrics  of  Multi-Epoch  Analysis  carry  over,  including  the  NPT, 
fNPT,  eNPT,  efNPT,  and  Expense  Variability,  with  slight  differences  in  calculation. 
For  example,  the  NPT/fNPT  of  each  design  in  an  era  are  calculated  using  only  the 
epochs  in  that  era,  and  a  Pareto  trace  can  also  be  calculated  as  a  time-weighted 
average  across  the  era.  Once  calculated  for  each  design  in  each  era,  the  NPT 
values  can  be  compared  in  various  ways.  One  could  look  for  the  extreme  NPT 
values,  whether  high  or  low,  leading  back  to  Process  8  and  observation  of  the 
properties  and  interactions  of  the  specific  design  and  era  that  cause  these  extreme 
values.  (In  the  NGCS  case,  for  example,  it  might  be  discovered  through  this  activity 
that  Design  #3  performs  very  poorly  in  eras  with  long  durations  of  the  Conflict  epoch. 
Further  investigation  could  lead  to  the  insight  that  the  lack  of  UAV  capacity 
drastically  reduces  satisfaction  in  eras  involving  Conflict.)  One  could  also  look  for  the 
highest  average  NPT  over  all  eras  to  identify  a  design  that  consistently  delivers 
value  across  all  identified  possible  futures. 

The  effective  versions  of  the  NPT  and  fNPT  can  become  more  informative  in 
this  process  as  well.  Because  transition  rules  and  costs  can  depend  directly  on 
contextual  uncertainties  (whether  epoch  variables  or  stakeholder  needs),  the 
eNPT/efNPT  metrics  can  help  identify  path-dependent  weaknesses  of  particular 
change  strategies.  The  eNPT/efNPT  metrics  of  Process  6  only  considered  transition 
costs  and  rules  for  one  epoch,  but  analyzing  these  same  metrics  through  sequences 
of  epochs  can  reveal  particularly  devastating  (or  beneficial)  contextual 
developments.  For  example,  in  the  NGCS  case,  assume  Design  #3  started  in  the 
Baseline  epoch,  followed  by  a  change  to  Design  #5  in  the  Mothership  epoch  based 
on  the  rule  “Maximize  Efficiency.”  That  sequence  appears  to  satisfy  stakeholder 
needs  very  well  relative  to  the  other  designs.  If  the  Mothership  epoch  were  followed 
by  the  Conflict  epoch,  however,  and  no  change  was  allowed  from  Design  #5  to  any 
design  on  the  Pareto  front — whether  due  to  the  transition  budget  or  established 
transition  rules — the  eNPT  of  Design  #3  in  the  era  would  be  resultantly  lower  (how 
much  lower  depends  on  the  duration  of  the  epochs  and  era).  This  consequence  is 
only  one  example  of  many  path-dependent  behaviors  that  emerge  from  the 
incorporation  of  change  strategies  over  the  long  run. 

As  noted,  the  Expense  Variability  metric  can  also  be  applied  during  this 
process,  identifying  contextual  sequences  that  provide  for  either  stable  or  turbulent 
expenses  relative  to  other  eras.  The  goal  of  all  the  analyses  of  Process  9  is  to 
identify  those  designs  and  change  strategies  that  result  in  efficient  value  delivery  at 
low  and  stable  costs  throughout  the  entire  system  lifecycle. 

Discussion  of  Impact  on  Development  Cycle 

The  epoch  variables  analyzed  thus  far  in  the  current  study  have  focused 
primarily  on  the  post-acquisition  phases  of  the  system  lifecycle  (e.g.,  technology 
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levels  of  UAVs,  emissions  regulations,  mission  parameters).  One  natural  extension 
of  the  EEA  approach  is  to  model  the  epoch  variables  of  the  development  cycle 
itself — that  is,  to  model  each  stage  of  development  in  one  or  more  possible  epochs 
comprised  of  budget  level,  material  and  labor  availability,  political  relations,  and  so 
forth.  Example  epoch  variables  in  such  a  study  are  shown  in  Figure  45. 


Notional  for  Development  Cycle 

Epoch  Variable 

[Range] 

Units 

Subsysteml 

[4,6,8] 

TRL  Level 

Labor  available 

[Nominal,  Low] 

Level 

Acq.  Budget 

[80%,  90%,  100%,  1 10%] 

Baseline  Adj. 

Ops  Budget 

[80%,  90%,  100%,  1 10%] 

Baseline  Adj. 

Figure  45.  Example  Epoch  Variables  With  Notional  Ranges  for  Describing 

Development  Cycle  Environments 

In  the  example  variables  shown  in  Figure  45,  several  types  of  resources  are 
modeled:  subsystem  availability,  labor,  and  budgets.  These  types  of  variables  could 
be  modeled  at  any  level  of  detail  desired — instead  of  an  overall  budget  number,  for 
example,  colors  of  money  could  be  used  to  represent  specific  capability  or 
operations.  The  epochs  formed  by  the  combinations  of  these  variables’  levels  could 
then  be  sequenced,  creating  eras  describing  possible  developments  of  the 
environment  throughout  preliminary  design,  detailed  design,  and 
construction/production.  Such  an  approach  could  efficiently  encapsulate  much  of  the 
uncertainty  present  throughout  the  long  development  lives  of  major  systems  and 
programs. 

This  encapsulation  would  provide  several  strategic  benefits  for  decision¬ 
makers.  First,  it  would  allow  easy  identification  of  systems  that  require  total 
commitment — those  with  no  adequate  “fall-back”  plan  in  the  case  of  falling  budgets, 
infeasible  subsystems,  or  other  resource  perturbations  that  might  occur  before  the 
system  production  and  use.  Second,  and  inversely,  it  would  help  identify  system 
choices  that  would  limit  expansion  of  quantity  or  capabilities,  should  such  expansion 
be  desired  in  the  case  of  a  new  threat  or  political  development  (e.g.,  the  increasing 
budgets  in  Figure  45).  Third,  decision-makers  could  potentially  identify  the  common 
capabilities  of  a  number  of  highly  desirable  systems,  allowing  production  of  those 
core  capabilities  to  begin  while  deferring  the  final  configuration  decisions  until  later  in 
the  development  cycle,  when  the  system’s  initial  operational  context  can  be  more 
accurately  determined. 

Affordability  Insights 

As  noted  in  the  introduction  to  this  case,  the  application  of  this  design  method 
directly  informs  the  analysis  of  system  affordability  for  the  potential  NGCS  designs. 
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By  using  the  MAE  function,  the  stakeholder  preferences  on  various  resources  are 
combined:  development  schedule  (IOC)  with  monetary  (e.g.,  acquisition  and 
lifecycle)  and  non-monetary  (e.g.,  labor)  expenses.  Using  this  metric,  it  can  be  seen 
that  Designs  #2,  #4,  and  #5  all  have  similar  resource  expenditures  in  most  epochs, 
while  Designs  1  and  6  are  somewhat  higher.  This  measurement  alone  could  provide 
a  good  starting  point  for  selecting  Designs  #2,  #4,  and  #5  as  designs  of  interest.  The 
Max  Expense  metric  from  Process  6  shows  that  Design  #2  could  require  slightly 
more  crew  (265)  than  Designs  4  and  5  (260  and  255),  but  Design  #2  could  also  cost 
less  (albeit  not  by  much)  over  its  lifecycle  ($6.1  billion  vs.  $6.3  billion).  Finally,  the 
Expense  Variability  metric  from  Process  6  shows  that  these  designs’  expenses  vary 
in  similar  ways  over  the  epochs.  Since  all  three  of  these  designs  appear  to  be 
affordable  alternatives,  other  measures  (such  as  stakeholder  satisfaction)  can  be 
used  to  further  restrict  the  number  of  designs  under  consideration.  Observing  Design 
#5’s  drop  in  utility  in  the  Conflict  epoch  (see  Era  #2,  for  instance)  could  result  in  it 
being  removed  from  consideration,  since  it  has  the  same  level  of  expense  as  Design 
4,  which  has  more  stable  value  delivery  throughout  Conflict  and  other  epochs. 
Designs  #2  and  #4  can  be  further  examined,  then,  to  identify  the  design  variables 
and  attributes  that  enable  these  to  be  affordable  choices  in  the  design  of  the  NGCS 
system.  For  example,  both  designs  are  a  length  of  520-530  feet  with  medium  levels 
of  Anti-Surface  and  Anti-Aircraft  capabilities.  These  common  traits  could  be  further 
investigated  to  gain  insight  into  the  interactions  between  these  particular  variables, 
the  ship  attributes  they  most  influence,  and  the  stakeholder  preferences  on  those 
attributes.  In  this  way,  the  affordability  analysis  performed  through  the  application  of 
this  method  leads  into  a  study  of  the  common  traits  of  affordable  NGCS  solutions. 
Rather  than  pointing  to  one  solution  deemed  “most  affordable,”  this  approach 
provides  stakeholders  with  a  new  perspective  on  the  affordability  of  systems  during 
the  conceptual  design  phase,  allowing  them  to  better  understand  the  complex 
behaviors  of  the  system  across  environments  as  well  as  the  trades  at  play. 

Discussion 

As  affordability  remains  a  relatively  new  ility,  there  are  many  variations  of  its 
definition  and  analysis.  Most  definitions  of  affordability  are  concerned  with  the 
balance  of  performance,  cost  and  schedule  needs,  and  constraints  over  the  system 
lifecycle.  However,  unlike  other  system  ilities,  performance  is  no  longer  regarded  as 
sine  qua  non  in  affordability  considerations.  In  the  paradigm  of  designing  for 
affordability,  systems  are  not  only  architected  for  performance  and  risk  mitigation, 
but  also  with  explicit  cost  and  schedule  considerations.  Affordability  has  emerged  as 
a  high  priority  ility  that  directs  the  early  stage  design  process  towards  developing 
systems  with  greater  cost  effectiveness  and  schedule  effectiveness.  Recent 
management  failures  in  high-profile  defense  programs  have  further  underlined  the 
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importance  of  affordability  considerations  during  early-phase  design.  This  motivates 
the  research  on  a  method  for  affordability  tradeoffs  under  uncertainty. 

An  important  task  within  the  conducted  research  was  a  literature  study,  along 
with  interviews  of  selected  experts.  This  included  reviewing  various  definitions  and 
analytic  frameworks  that  have  been  proposed  to  incorporate  affordability 
considerations  into  current  systems  engineering  practice.  Despite  advances  in  the 
literature,  two  key  challenges  were  identified  in  formalizing  a  paradigm  of  designing 
for  affordability: 

1.  Lack  of  an  accepted  definition  and  a  set  of  guiding  principles  for 
affordability  within  the  systems  community.  This  challenge  has 
resulted  in  a  plethora  of  proposed  affordability  approaches,  with  a 
difficulty  in  directly  comparing  these  approaches  due  to  underlying 
differences  in  the  meaning  of  what  makes  an  “affordable”  system. 

Once  a  common  definition  and  a  common  set  of  principles  for 
affordability  are  identified  by  the  community,  efforts  can  be  made  to 
integrate  approaches  taken  by  the  government,  industry,  and 
academia  into  a  concerted  effort  for  reducing  overall  program  costs 
and  schedule  slippages. 

2.  Absence  of  mature  metrics  and  systematic  frameworks  for 
comprehensive  affordability  analysis.  Traditional  cost-related 
metrics  have  been  proposed  for  affordability  analyses,  but  have  been 
deemed  insufficient  to  encompass  the  broader  intent  of  current  design 
for  affordability  efforts.  Other  metrics,  such  as  the  use  of  a  multi¬ 
criterion  cost  function,  are  less  mature  and  warrant  further  study  to  test 
for  generality  and  feasibility  in  actual  practical  analyses.  A  systematic 
framework  with  an  accompanying  set  of  metrics  would  enable 
affordability  analysis  to  be  conducted  consistently  and  more  holistically 
than  just  simply  extending  prior  practice  (e.g.,  analysis  must  include 
aspects  of  acquisition  cost,  operations  and  maintenance  cost, 
development  schedule,  upgrade  schedule,  externality  costs,  and 
available  budgets,  all  over  time). 

While  performance  is  easily  quantified  through  technical  specifications,  cost 
and  schedule  are  much  harder  to  measure  objectively  and  accurately.  Many 
quantitative  methods  have  been  proposed  to  conduct  affordability  analysis,  ranging 
from  probabilistic  estimates  of  cost  and  schedule  risk  to  setting  either  cost  or  time  as 
independent  variables  during  engineering  tradeoffs.  In  the  search  for  affordable 
solutions,  this  research  proposed  the  use  of  tradespace  exploration-related  methods 
to  facilitate  the  selection  and  identification  of  architectures  that  best  fit  cost  and 
schedule  requirements  of  stakeholders.  The  Multi-Attribute  Tradespace  Exploration 
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(MATE)  method  can  be  used  to  perform  such  affordability  tradeoffs,  as  it  allows  the 
capturing  of  stakeholder  preferences  for  simultaneous  multiple  objectives. 

MATE  resolves  conflicting  and  subjective  evaluations  of  decision-making 
processes  by  combining  various  single-attribute  utility  functions  for  every  attribute  of 
interest  into  a  single  function  that  quantifies  how  a  decision-maker  values  different 
attributes  relative  to  one  another.  To  effectively  evaluate  the  impact  of  dynamic 
variation  in  costs  with  tradeoffs  in  decision  parameters  and  time  to  completion,  this 
research  has  applied  Epoch-Era  Analysis  (EEA),  as  this  method  considers  and 
clarifies  the  effects  of  changing  contexts  and  needs  over  time  on  the  perceived  value 
of  a  system  in  a  structured  manner.  Instead  of  discretizing  the  system  lifecycle 
according  to  traditional  system  milestones,  EEA  discretizes  the  lifecycle  according  to 
impactful  changes  in  the  operating  environment  (e.g.,  available  budget), 
stakeholders,  or  the  system  itself,  through  the  constructs  epochs  and  eras.  Epochs 
represent  time  periods  with  fixed  contexts  and  needs,  while  eras  are  sets  of 
sequentially  ordered  epochs.  This  method  would  allow  for  the  consideration  of 
changes  to  a  system  throughout  multiple  possible  alternative  lifecycles  and  allow  for 
the  identification  of  systems  that  are  potentially  more  efficient  in  resource  costs. 

In  the  application  of  these  methods,  this  research  has  proposed  the  use  of  a 
new  metric,  Multi-Attribute  Expense  (MAE),  to  quantify  acceptability  of  resource 
usage,  where  resources  include  multiple  distinct  criteria  including  cost  and  time 
parameters  of  a  system.  An  MAE  function  assigns  weights  to  different  cost  and  time 
parameters  in  the  same  way  as  a  multi-attribute  utility  function.  The  MAE  metric 
reflects  the  degree  of  stakeholder  satisfaction  with  resources  expended  in  the  design 
or  operation  of  a  system.  Augmented  by  resource  metrics,  MATE  and  EEA  can  be 
combined  to  form  a  systematic  method  for  architecting  affordable  solutions  since 
achieving  affordability  requires  good  decisions  both  upfront  (MATE),  as  well  as 
across  the  lifecycle  (EEA).  This  method  has  been  applied  to  a  case  study  involving 
the  design  of  a  next-generation  combat  ship  (NGCS)  to  validate  the  method’s 
feasibility. 

As  shown  in  the  NGCS  case  study,  the  application  of  the  proposed  method 
leads  to  some  salient  points  about  affordability  analysis  in  the  conceptual  design 
phase.  First,  capturing  expenses  as  attributes  preserves  stakeholder  and  model 
information,  as  opposed  to  losing  information  through  monetization  of  non-monetary 
expenses  (e.g.,  dollars  per  ton  CO2).  For  example,  Years  to  Initial  Operating 
Capability  (IOC)  of  each  NGCS  design  is  not  easily  converted  to  a  monetary 
amount,  but  it  can  be  successfully  incorporated  into  the  MAE.  The  second  point  is 
that  using  EEA  can  facilitate  better  understanding  of  a  design’s  affordability  over  its 
entire  lifecycle,  as  practitioners  can  quickly  discern  how  expensive  an  alternative 
might  be  with  regard  to  levels  of  resource  usage  (i.e.,  Max  Expense)  as  well  as  how 
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resource  usage  may  change  across  multiple  epochs  and  multiple  eras  (i.e.,  Expense 
Variability).  If  budget  levels  are  established  a  priori  for  acquisition  costs,  operating 
costs,  labor  usage,  development  schedule,  or  other  expenses,  the  Max  Expense  and 
Expense  Variability  metrics  can  be  compared  against  those  levels  to  identify 
affordable  solutions.  In  the  absence  of  any  established  levels,  potential  designs  can 
still  be  compared  against  one  another  in  the  tradespace. 

Through  this  method’s  approach  to  tradespace  exploration  and  epoch-era 
analysis,  decision-makers  can  be  more  informed  of  the  risks  associated  with  each 
potential  design’s  resource  needs,  and  hence  gain  further  insights  into  each 
potential  design’s  affordability  relative  to  other  available  alternatives. 

This  research  project  has  made  progress  in  addressing  the  two  challenges 
identified  in  the  literature  review  and  conversations  with  selected  experts:  lack  of 
accepted  definitions  and  lack  of  metrics  with  systematic  framework. 

Lack  of  accepted  definitions.  The  first  challenge  identified  in  this  research  is  the 
lack  of  accepted  definitions.  The  research  met  this  challenge  through  the  following: 

•  Proposed  a  definition  for  affordability.  After  identifying  the  major 
themes  in  affordability  analysis  and  considering  them  in  light  of  related 
analysis  approaches,  affordability  can  be  defined  as  the  property  of 
becoming  or  remaining  feasible  relative  to  resource  needs  and 
resource  constraints.  Resource  needs  are  the  cost  and  schedule 
preferences  elicited  from  stakeholders,  and  resource  constraints  are 
the  external  restrictions  imposed  upon  these  preferences  that  limit  the 
range  of  feasible  solutions.  Feasible  here  means  not  violating 
acceptability  bounds,  which  include  both  constraints  and  minimum 
acceptable  need  levels.  For  example,  a  program  may  desire  a  $100 
million  unit  cost  per  vehicle,  while  given  a  budget  of  $300  million.  If  the 
unit  cost  is  less  than  $100  million,  that  is  better  for  the  program.  In  fact, 
the  program  prefers  that  the  unit  cost  is  a  low  as  possible.  In  this 
particular  case,  the  system  affordability  increases  as  it  remains  below 
$300  million  and  unit  costs  drop.  As  systems  and  operating  contexts 
are  dynamic,  resource  needs  and  resource  constraints  may  change 
over  time,  therefore  affordability  for  a  given  system  may  change  as 
well. 

•  Proposed  a  definition  for  an  affordable  design  solution.  Given  the 
above  definition  of  affordability,  an  affordable  design  solution  is  one 
that  is  feasible  when  it  fulfills  resource  needs  and  functions  within  the 
resource  constraints  for  a  fixed  context.  As  contexts  change,  a 
particular  solution  may  remain  in,  enter,  or  exit  the  feasible  set  of 
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solutions.  Affordable  solutions  are  those  that  remain  in  or  enter  the 
feasible  set  of  solutions. 


•  Proposed  affordability  analysis  to  be  conducted  via  the 

identification  of  solutions  that  remain  affordable  for  part  (short 
run)  or  all  (long  run)  of  the  system  lifecycle.  With  this  guiding 
principle,  an  affordable  solution  that  is  identified  will  be  capable  of 
satisfying  possible  changing  resource  requirements  and  resource 
constraints  over  the  system  lifecycle.  Affordability  can  then  apply  to 
both  short  term  (meeting  affordability  bounds  across  one  or  several 
contexts)  as  well  as  long  term  (meeting  affordability  bounds  across 
many  or  all  experienced  contexts). 


Lack  of  metrics  with  systematic  framework.  The  second  challenge 
identified  in  this  research  is  the  lack  of  metrics  with  a  systematic  framework.  This  is 
how  the  research  addressed  that  challenge: 

•  Proposed  method  incorporating  MATE,  EEA,  and  MAE  to  conduct 
affordability  analysis.  The  research  has  leveraged  related  existing 
methods  to  address  the  challenges  identified  in  the  literature,  including 
systematic  consideration  of  alternatives  during  up  front  system 
selection  (MATE),  throughout  operations  and  maintenance  (EEA),  as 
well  as  through  the  incorporation  of  a  multi-criteria  perspective  on 
resource  use  (MAE). 

•  Applied  method  to  demonstration  case  (NGCS).  This  research  has 
applied  the  proposed  the  method  to  the  NGCS  case  study  as  a 
preliminary  validation  of  the  method’s  feasibility. 

Existing  methods  for  conducting  affordability  analysis  are  relatively  new,  and 
the  application  of  expense  functions  and  tradespace  exploration  methods  constitute 
yet  another  proposed  approach  to  search  for  affordable  solutions.  The  addition  of 
another  definition  and  another  affordability  method  does  not  solve  the  upfront  pair  of 
challenges  identified,  but  rather  adds  to  the  potential  solution  space  for  what  may  be 
most  useful  to  the  community.  This  research  constitutes  a  step  in  trying  to 
synthesize  some  of  the  key  themes  discussed  by  the  community,  while  proposing  an 
alternative  approach  that  does  not  rely  solely  on  cost-modeling  techniques.  Further 
work  to  validate,  refine,  and  socialize  the  concepts  in  this  research  are  essential  for 
fostering  consensus  in  the  community. 

While  the  application  of  this  new  method  has  been  demonstrated  in  a  simple 
case  study,  it  has  yet  to  be  extended  to  more  complex  case  studies  with  much  larger 
sets  of  design  solutions.  It  could  be  useful  to  validate  the  proposed  method  using 
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additional  case  studies,  as  well  as  to  conduct  a  method  comparison  of  this  approach 
to  other  existing  approaches. 

Following  the  research  conducted  in  this  project,  a  number  of  potential  next 
step  activities  have  been  identified: 

•  Extending  the  method  to  more  complex  case  studies  with  more 
design  solutions.  To  ensure  that  affordability  analysis  can  be 
conducted  more  comprehensively,  research  should  be  done  to  validate 
the  feasibility  of  the  method  in  a  wider  variety  of  real-world 
applications. 

•  Extending  affordability  analysis  from  a  system  to  a  project, 
program,  and  portfolio.  As  a  project,  program,  and  portfolio  can  differ 
greatly  in  terms  of  both  scale  and  scope,  affordability  analyses 
conducted  at  these  levels  cannot  only  derive  affordable  architectural 
solutions,  but  must  also  generate  recommendations  for  affordable 
acquisition  strategies  and  policy  implementations. 

•  Exploring  the  relationship  of  affordability  with  other  system  ilities 
like  flexibility  and  survivability.  Research  can  be  done  to  identify 
affordably  changeable  or  affordably  survivable  solutions  by  revisiting 
case  studies  previously  conducted  for  the  understanding  of  the 
aforementioned  ilities.  Quantitative  metrics  for  flexibility  and 
survivability  can  be  used  in  concert  with  the  newly  proposed 
affordability  metrics  and  analysis  method  for  affordability. 
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